 MSDN Magazine November 2005

MSDN Magazine November 2005

Are You in the Know?: Find Out What's New with Code Access Security in the .NET Framework 2.0

Unlike role-based security measures, code access security is not based on user identity. Instead, it is based on the identity of the code that is running, including information such as where the code came from. Here Mike Downen discusses the role of code access security (CAS) in .NET and outlines some key new features and changes in CAS for the .NET Framework 2.0.

Mike Downen Do You Trust It?: Discover Techniques for Safely Hosting Untrusted Add-Ins with the .NET Framework 2.0

When you allow your application to run arbitrary code through an add-in, you may expose users to unknown code, running the risk that malicious code will use your application as an entry point into the user's data. There are several techniques you can use to reduce the attack surface of your application, which Shawn Farkas discusses here.

Shawn Farkas Are You Protected?: Design and Deploy Secure Web Apps with ASP.NET 2.0 and IIS 6.0

Ensuring the security of a Web application is critical and requires careful planning throughout the design, development, deployment, and operation phases. It is not something that can be slapped onto an existing application. In this article, Mike Volodarsky outlines best practices that allow you to take advantage of the security features of ASP.NET 2.0 and IIS 6.0 to build and deploy more secure Web applications.

Mike Volodarsky Who Goes There?: Upgrade Your Site's Authentication with the New ASP.NET 2.0 Membership API

Here Dino Esposito and Andrea Saltarello cover the plumbing of the Membership API and its inherently extensible nature, based on pluggable providers. To demonstrate the features, they take an existing ASP.NET 1.x authentication mechanism and port it to ASP.NET 2.0, exposing the legacy authentication mechanism through the new Membership API.

Dino Esposito and Andrea Saltarello What Gives You the Right?: Combine the Powers of AzMan and WSE 3.0 to Protect Your Web Services

In this article, Niels Flensted-Jensen demonstrates how you can combine new and existing Microsoft technologies with minimal new code to provide flexible authorization for individual Web service methods. Windows 2003 Authorization Manager, Web Service Enhancements 3.0, and Enterprise Library all play a part.

Niels Flensted-Jensen How Do They Do It?: A Look Inside the Security Development Lifecycle at Microsoft

In this article, Microsoft security expert Michael Howard outlines how to apply the Security Development Lifecycle to your own software development processes. He explains how you can take some of the lessons learned at Microsoft when implementing SDL and use them in your own development process.

Michael Howard Editor's Note: Many Levels of Security

Every year at this time, we bring you our now-famous security issue. We recognize the vast importance of writing and deploying secure code—it affects so many areas of concern—which is why we devote an entire issue each year to the topic.

New Stuff: Resources for Your Developer Toolbox

Code refactoring is defined as the act of changing code without changing what it does. The actual work involved in refactoring—extracting methods from oversized modules, tidying up and unifying variable names, identifying unhandled exceptions, and all the other cleanup, simplification, and standardization chores—can be daunting, indeed, and risky too.

Marnie Hutcheson Web Q&A: ASP.NET Session State, Validation, DataGrids, and More

Edited by Nancy Michell Data Points: DataSet and DataTable in ADO.NET 2.0

ADO. NET 2. 0 sports some exciting enhancements to the core classes found in ADO. NET 1. x and introduces a variety of new classes, all of which promise to improve performance, flexibility, and efficiency.

John Papa Basic Instincts: Programming I/O with Streams in Visual Basic .NET

If you are just now migrating from Visual Basic® 6. 0 to Visual Basic . NET, be prepared for a change, because the way you program file I/O is very different. The good news is that an idiosyncratic, proprietary approach has been replaced by an elegant and reusable one that has become an industry standard.

Ted Pattison Cutting Edge: A Quick Tour of Themes in ASP.NET 2.0

It's much easier to build a rich user interface into your Web application in ASP. NET 2. 0 than it was in previous versions. Master Pages let you build pages based on existing templates of markup and code.

Dino Esposito Service Station: An XML Guru's Guide to BizTalk Server 2004, Part I

Who would have believed that XML, such a seemingly trivial technology, could revolutionize an industry? It may have seemed like a long shot in the beginning, but the XML aficionados saw something special and pragmatic right away—a sort of duct tape for the world's information systems.

Aaron Skonnard Bugslayer: SUPERASSERT Goes .NET

Those of you who have been reading this old Bugslayer column over the last nine years have branded into your frontal lobe a single word: ASSERT! Anytime you can have the code tell you about a problem instead of having to find it by slaving away with a debugger is a huge timesaver.

John Robbins Security Briefs: Security Features in WSE 3.0

I've been spending a lot of time lately building secure Web services with the Microsoft® . NET Framework 2. 0, and Web Services Enhancements (WSE) 3. 0 has been a lifesaver for me, so I thought it would be appropriate to dedicate a column to security features in this new product.

Keith Brown C++ at Work: Disabling Print Screen, Calling Derived Destructors, and More

Paul DiLascia { End Bracket }: Phoenix Rising

Phoenix is neither a compiler nor a JITer, but will eventually transform both. It is the codename for an internal Microsoft project that provides an extensible framework for the analysis, optimization, and modification of code during compilation.

Guy Eddon


Are You in the Know?: Find Out What's New with Code Access Security in the .NET Framework 2.0

Are You in the Know?

Find Out What's New with Code Access Security in the .NET Framework 2.0

Mike Downen
This article is based on a prerelease version of the .NET Framework 2.0. All information herein is subject to change.
This article discusses:
  • Overview of CAS
  • Sandboxing and trust levels
  • Developing hosts and frameworks
  • AppDomains and security
This article uses the following technologies:
.NET Framework 2.0, Visual Studio 2005
Contents
Why CAS?
Understanding Sandbox Permissions
Hosts and Frameworks
AppDomains and Security
Defining the Sandbox
How to Host
CAS and Frameworks
Security-Transparent Code
Using Transparency
Conclusion

S
ecurity in the Microsoft® .NET Framework encompasses many technologies: role-based security in the base class libraries (BCL) and in ASP.NET, cryptography classes in the BCL, and new support for using access control lists (ACLs) are just a few examples. One of the technologies in the .NET security spectrum provided by the common language runtime (CLR) is code access security (CAS). This article discusses the role of CAS in .NET security and some key new features and changes in CAS in the .NET Framework 2.0.

Most developers using the .NET Framework don't need to know much about CAS except that it exists. As with floating point numbers, CAS is a feature of the Framework about which intricate and detailed knowledge can be very useful for some applications but that most can largely do without.

Why CAS?

When people ask me about access control, they often want to know more about role-based security—controlling access to resources based on user identity. CAS can be difficult to understand because it is not based on user identity. Instead, it is based on the identity of the code that is running, including information such as where the code came from (for example, from the local computer or from the Internet), who built it, and who signed for it. The actions the code can perform and the resources the code can access are restricted based on this "evidence" associated with the code and its identity.

Why would you want to restrict the things code can do based on its identity? Often you don't. For example, you may run a fancy and expensive graphics editing program that can access any resource on the computer that you, as the user running it, can access (files, registry settings, and the like). In this case, you know and trust the publisher of the software and are willing to allow it a high level of access to the platform on which it is running. While you typically want an app to run with the least-privileged user account necessary in order to prevent accidental corruption of the computer, trust of the application itself is not an issue in this case.

However, there are situations in which you run code and either do not know or do not fully trust its author. The most common example of this today is in browsing the Web. Modern browsers often run code from the Web sites they visit. This code is usually JavaScript or DHTML, but it could be some other form of executable recognized by your browser. When browsers run this code on your computer, they do it in a sandbox—a restricted execution environment that controls which resources the code can access on your computer. This sandbox is a necessity; otherwise, relatively anonymous code from the Internet would be able to read your private files and install viruses on your computer every day. Many browsers offer different levels of sandboxing based on the origin of the code that is running. For example, Microsoft Internet Explorer defines the concept of zones. Code from the Internet zone cannot do as much as code from the Trusted sites zone. Security exploits occur when either a bug exists in the software enforcing the sandboxing rules or the user is tricked into approving code for execution outside the sandbox.

Running code from a Web site in your browser is a common scenario for sandboxing and code identity-based security. CAS in the CLR provides a general mechanism to sandbox code based on code identity. This is its main strength and the main scenario for its use. CAS is used today by Internet Explorer to sandbox managed code from Web sites running in the browser (either controls or standalone apps). It's used to sandbox applications running from a network share on your local intranet. It is also used by Visual Studio® Tools for Office (VSTO) in order to host managed add-ins for Microsoft Office documents. In server scenarios, CAS is used by ASP.NET for Web applications and by SQL Server™ 2005 for managed stored procedures. CAS pops up in lots of scenarios, usually behind the scenes.

Understanding Sandbox Permissions

Developers often ask what they need to know about CAS. If you're an app developer, the answer depends on whether you are writing an app that will run in a sandbox, such as a managed control in Internet Explorer. If so, you need to know the following:

  • What you can do in the sandbox
  • Whether that is enough functionality for the application to run successfully
  • How to elevate the trust of the application to get more permissions if necessary

For example, managed controls in the browser use a default set of sandbox permissions: the Internet Permission Set. In terms of resource access, this sandbox allows the app to create "safe" user interface elements (transparent windows are considered unsafe, for example, as they could be used to commit spoofing or man-in-the-middle attacks), make Web connections back to its site of origin, get access to the file system and printers with user consent, and store a limited amount of data in isolated storage (similar to cookies for Internet Explorer). It does not allow an app to read random parts of the file system or registry, or environment variables. Nor does the sandbox allow the app to create connections to a SQL Server database, or to call COM objects or other unmanaged code. Different hosts may define different sandboxes. For example, SQL Server and ASP.NET each define a different set of permissions for their low-trust apps. Figure 1 outlines these sandbox permissions.

Figure 1 Permissions of Various Sandboxes (Close Figure 1)

Sandbox Resource | Internet Permission Set | LocalIntranet Permission Set | ASP.NET Minimal Trust Permission Set
Execute code | Yes | Yes | Yes
User interface | Safe top-level windows | Unrestricted | None
File system | Isolated Storage: isolated by user with a quota. File Dialog: can access data in files chosen by the user. | Isolated Storage: isolated by user but no quota. File Dialog: can access data in files chosen by the user. | None
Network | Can make connections to site of origin only. | Can make connections to share or site of origin only. Can access DNS. | None
Printing | Can print only if allowed by the user. | Programmatic access to the default printer and to other devices if allowed by the user. | None
Environment | None | USERNAME environment variable. | None
Reflection | Public reflection. | Public reflection and can emit types. | None
Registry | None | None | None
Database | None | None | None
Win32 APIs or COM objects | None | None | None

Once you know the options for a sandbox, you can decide whether a sandbox app is the right choice for your needs. For example, if you're writing a managed control in Internet Explorer and are planning to connect to a SQL database, your control would not be able to run in the sandbox by default. You would either have to change your plans (for instance, by accessing data using a Web service deployed on the control's server), or elevate the trust level of your control on the client computer. For managed controls in the browser, elevating trust involves deploying CAS policy to trust both the site the control comes from and the control itself. Other hosts have different mechanisms for trusting apps.To change the trust level of an ASP.NET app, you must change a setting in a configuration file.

In some cases, your app must run in the sandbox because it is unlikely your users would install and run it any other way. Suppose you're writing a simple survey control for a Web site. The control asks the user a few questions and records their responses. It is unlikely most people would download and run a full-trust application for something so simple. However, there are some cases in which you're writing a trusted app, but still benefit from having it run in the sandbox. Figure 2 shows the levels of trust with and without sandboxing. And if your application must run with full trust, make sure it still behaves well in partial-trust environments (see the sidebar "Handling Partial Trust Gracefully" for more information).

Figure 2 Sandbox Trust Scenarios (Close Figure 2)

 | Sandbox | No Sandbox
Trusted | Typical benefits include easier installation, better reliability, lower total cost of ownership. | Typical scenario for client code—app requires a high level of access to the platform.
Not trusted | You get better security—a safe way to run low-trust code. | A dangerous scenario—code should not run.

Typically, it's much easier to deploy a sandbox application. With ClickOnce, for example, sandbox applications just run and do not require any action from the user, while high-trust applications must either be signed with a trusted certificate or show a prompt to the user (for more information about ClickOnce, see ClickOnce: Deploy and Update Your Smart Client Projects Using a Central Server and Escape DLL Hell: Simplify App Deployment with ClickOnce and Registration-Free COM).

You may also get reliability benefits from sandboxing applications. For example, running as many managed stored procedures as possible in the lowest trust level of SQL Server means there's less code that has full access to the server and, therefore, less code that can damage the server (whether maliciously or accidentally).

Visual Studio 2005 provides much improved support for building sandbox client applications to be deployed using ClickOnce. The Security pane of the project properties page is where most of the action takes place (see Figure 3). Here, you enable security settings and set the zone you're targeting for your application. For a sandbox app, you'd typically choose the Internet zone. You can then use the Calculate Permissions button to give you a rough estimate of the permissions required for your application, and whether it fits in the sandbox (this calculation functionality is also available from the permcalc command-line tool included with the .NET Framework 2.0 SDK).

Figure 3 Visual Studio 2005 Security Pane
Figure 3 Visual Studio 2005 Security Pane

It's important to understand that this is a rather rough, usually conservative, estimate from purely static analysis. You should always test your application with the permissions with which it will run. You do this by simply debugging the application after setting the target zone. Visual Studio will run the application with the permissions you specified on the security pane.

Hosts and Frameworks

If you're writing hosts or frameworks, you need to know much more than just the basics about the sandbox. Host developers need to know how to host low-trust code. Framework developers need to know how to write frameworks or libraries that expose functionality to low-trust code.

However, before I get into those issues, I want to discuss a topic of common confusion—running with least privilege. Running with least privilege refers to running code with the fewest privileges or permissions required to get the job done. This limits the damage that can be caused if there is an exploitable bug in the code. For Windows® security, running with the least-privileged user identity possible (for example, a normal user instead of an admin user) is always a good idea. For CAS, running with least privilege means running with the minimal set of CAS permissions required for an application or library to perform its tasks. While running with least privilege is generally a good thing, it does have its limits. For example, almost all host or framework code must run with some powerful privileges that are outside of the default sandbox. This code frequently has to call Win32® APIs or other unmanaged code and control resources on the machine such as files, registry keys, processes, and other system objects. Since this kind of code already requires elevated permissions, trying to remove a permission or two is generally not worth the effort from a security point of view, given that many powerful permissions can be elevated to full trust.

Your time is better spent auditing and testing the code to make sure it is secure in the face of hostile callers. (In the context of CAS, running with full trust means the code can do whatever the user running it could do on the system. If the process running full-trust code does not have admin privileges, even full trust code does not have unlimited access to the machine's resources.) In order to make that easier, the .NET Framework 2.0 includes a new technology called security-transparent code, which I'll discuss later in this article with other framework development issues. But first, let's get back to hosting.

In general, hosting means using the CLR to execute code in the context of another application. For example, SQL Server 2005 hosts the runtime to execute stored procedures written in managed code. I'm going to focus here on the aspects of hosting relevant to security, starting with a review of AppDomains. There is a lot more than that to hosting, however—see Stephen Pratschner's book Customizing the Microsoft .NET Framework Common Language Runtime (Microsoft Press®, 2005) for further details.

AppDomains and Security

AppDomains provide sub-process isolation for managed code. This means each AppDomain has its own set of state. Verifiable code in one AppDomain cannot tamper with code or data in another AppDomain unless interfaces created by the hosting environment allow them to interact. How does this work? Verifiably type-safe code (produced by default by both the C# and Visual Basic® .NET compilers) cannot access memory in arbitrary ways. Each instruction is checked by the runtime using a set of verification rules to make sure it accesses memory in a type-safe way. Therefore, the runtime can guarantee the isolation of AppDomains when running verifiable code, and it can prevent code from running if it's not verifiable.

This isolation allows a host to run code with different trust levels in the same process safely. Low-trust code can run in a separate AppDomain from either trusted host code or other low-trust code. The number of AppDomains needed for hosting low-trust code depends on the isolation semantics you want for your host. For example, Internet Explorer creates one AppDomain per site for managed controls. Controls from one site are able to interact in the same AppDomain, but cannot interfere with (or take malicious advantage of) controls from another site.

Handling Partial Trust Gracefully

By default, managed applications that run from Internet Explorer—either managed controls in the browser or no-touch deployment applications—run with a reduced set of CAS permissions. You can change CAS policy, however, to run specific trusted applications with more permissions. For example, you could trust a specific strong name when assemblies signed with it come from a specific URL. The .NET Framework Configuration Tool creates Microsoft Installer (MSI) files to do this. This security policy update would have to be installed on any machine running your application, either by instructing a user to run the MSI file themselves or having the MSI file deployed across your enterprise by your IT department.

If you write a managed application that runs from the browser but requires elevated permissions, you should check to make sure that your app or control gets the permissions it needs to run, and fails gracefully if it does not.

Why wouldn't your app have enough permissions? Perhaps a user forgot to install the security policy update before running your application. There are also cases when installing a new version of the .NET Framework would require the user to install updated security policy. For example, controls in the browser will always run against the latest version of the .NET Framework installed on the machine. CAS Policy, however, is specific to each version of the Framework. So if the user installed a policy update for version 1.1 of the Framework, and then later installed version 2.0, their control will now run against 2.0 and will not see the changes in policy applied to 1.1.

Making sure your application has the permissions it expects and helping the user fix the problem if it does not can help ensure that your controls will run in the future. You should make this permission check early in the life of your program. For example, if your application expects to run with full trust, you would add a method to your main form like the one shown in Figure A.

You would then change your application's Main method to something like the following:

[STAThread]
static void Main()
{
   if (CheckApplicationPermissions())
   {
      Application.Run(new Form1());
   }
}

The Web page to which you send the user for help could detect the latest version of the runtime installed on the machine (via the user agent header in the HTTP request) and provide the user with a policy update specific to their version of the runtime.

ClickOnce in the .NET Framework 2.0 makes this situation simpler for standalone apps. A ClickOnce application states its permission requirements in its manifest, and the application either runs with those permissions or does not run at all. So this permission-checking code is not necessary in a ClickOnce app, but is still useful for controls in the browser.

Figure A Checking For Full Trust (Close Figure A)
private static bool CheckApplicationPermissions()
{
   try
   {
      // See if we're running in full trust
      new PermissionSet(PermissionState.Unrestricted).Demand();
      return true;
   }
   catch (SecurityException)
   {
      try
      {
         // Not running in full trust, help the user
         MessageBox.Show("Application error: please go to" +
            "http://contoso.com/fix.aspx for fix instructions.");
      }
         // App doesn't even have permission to show UI, not much to do
      catch (SecurityException) {}
   }
   return false;
}
Figure 4 AppDomain Firewall
Figure 4 AppDomain Firewall

CAS allows you to specify the trust level of code as it loads (mapping evidence into permissions via policy). Additionally, CAS allows you to specify the trust level of AppDomains as they are created. In the .NET Framework 1.1, you can do this by setting evidence on the AppDomain. This evidence is mapped to a set of permissions by policy. Whenever the flow of control transitions into the AppDomain, the AppDomain's permission grant is pushed on the stack, just like the permission grants for assemblies, and is considered during any stack walks to evaluate demands from that AppDomain. This mechanism, called the AppDomain firewall (see Figure 4), prevents cross-AppDomain luring attacks and provides yet another isolation mechanism to isolate code between AppDomains.

CAS allows you to load code (assemblies) into an AppDomain with different trust levels (permissions), but this is not the recommended way to securely sandbox code. Given that all code in an AppDomain shares state, it is difficult to prevent elevations of privilege among multiple assemblies within the AppDomain. For example, say you load one assembly with permission to execute only and another assembly with the LocalIntranet permission set. It may be possible for the former (running with only permission to execute) to elevate to the LocalIntranet permission set, unless that code was written with extreme care for handling this scenario. You also lose the benefit of the AppDomain firewall. Thus, as I discussed, it's useful to create multiple AppDomains to host code that should be isolated, even if it is running at the same trust level, such as Internet Explorer does for controls from separate sites.

So instead of loading multiple assemblies into a single AppDomain at different trust levels, I recommend using separate AppDomains for each logical chunk of low-trust code (each site in Internet Explorer, for example), and keeping the security model simple. For each AppDomain, there should be two trust levels:

  • The first trust level is full-trust platform code, which includes the .NET Framework and the host's trusted code that interacts with the low-trust code.
  • The second trust level is low-trust code itself, running at one trust level, rather than mixed trust levels.

Figure 5 AppDomain Trust
Figure 5 AppDomain Trust

This model is easier to understand and more secure, which is one of the reasons why it is used by ClickOnce. For any ClickOnce app, there is code at two trust levels running in the AppDomain: the platform code running with full trust and the application code running with the trust level specified in the application's manifest. The CLR security system has been optimized for this mode, and as a result code adhering to this model will perform well. See Figure 5 for an illustration of both trust levels.

Defining the Sandbox

Another task in hosting is defining the sandbox, or set of permission, you'll grant to low-trust code. As I mentioned, the CLR defines some permission sets for sandboxes, as do other hosts (such as ASP.NET). The two relevant permission sets defined by the CLR are the Internet permission set and the LocalIntranet permission set. The Internet permission set is designed to be safe for running anonymous, potentially hostile code. It has been audited by Microsoft for this purpose and is the only set of permissions thoroughly tested for such a scenario. Therefore, it should be your default choice for hosting low-trust code. But while the LocalIntranet permission set will prevent hostile code from taking over a computer, it discloses some information users may not want anonymous code to have, such as the user name of the currently logged-in user. So it is actually more appropriate for somewhat trusted code, such as code on your organization's intranet.

What about hosts that have defined their own permission sets, such as ASP.NET? The ASP.NET medium-trust level, for example, provides security by using both CAS and Windows identity-based security to restrict code. One member of my team coined a good term for this kind of scenario: sand-duning. A "sand dune" is a set of restricted permissions that doesn't by itself prevent hostile code from unauthorized access to computing resources. Instead, it is useful when combined with other security enforcement mechanisms and for keeping honest code honest. It is a common technique for reliability in server scenarios wherein multiple apps are sharing computing resources and you want to prevent one poorly behaved app from being able to take down the server.

How to Host

How do you actually make all this happen? In the .NET Framework 1.1, you create an AppDomain, making sure to pass in evidence. This evidence will be used to compute the permissions for the AppDomain. Then you set up an AppDomain policy level, which should grant full trust to platform and host code, and grant low trust to all other code. Next, you set the AppDomain policy level on the AppDomain. Finally, you call the trusted host code in the new AppDomain to bootstrap running low-trust code.

In the .NET Framework 2.0, this process is simplified:

  1. Use the new simple sandboxing API to create an AppDomain. This sets the AppDomain trust level, the sandbox permission set, and the list of trusted host assemblies all in one step, without creating an AppDomain policy level.
  2. Call trusted host code in the new AppDomain to bootstrap running low-trust code.

For all the details on this, take a look at the Shawn Farkas article in this issue. Shawn explains how to do this and discusses all the techniques for doing it securely. While the simple sandboxing model is sufficient for many scenarios, you can have more control over the creation of AppDomains through the AppDomainManager class. Shawn's article talks about how you can use AppDomainManager to implement custom policy behavior, if you need it.

CAS and Frameworks

Now, I'm going to shift gears and discuss another developer scenario in which CAS is relevant—building frameworks. A framework is a library of reusable classes designed to be used by other applications. It can be one assembly with a few types, or a large set of assemblies with many types, like the .NET Framework. Frameworks are usually deployed in the Global Assembly Cache (GAC) so they can be used by multiple applications. They are platform components that run with full trust to take advantage of all that the system offers. This topic is also relevant to host developers for they usually must build at least a limited framework for hosted code to interact with, if not a full-fledged framework.

Any time a framework developer wants to expose functionality to code that does not have full trust, they must be aware of CAS. One example is building a sound library. You could define a custom permission, say SoundPermission, and demand it when your method to play a sound is called. If the demand succeeds, you would assert permission to call unmanaged code and call the Win32 API to play the sound.

An alternative and more common scenario is to interact with existing system permissions in building a framework. For example, let's say you're implementing a math library that is accessible from low-trust code. If your library only does calculations and does not access any system resources, it can largely ignore CAS. But suppose in the initialization of your math library, you read some environment variables to help you decide how to optimize your computations. You'll need to audit the code to make sure it is safe to call in the context of low-trust code. The call must be completely opaque to the low-trust code, meaning the low-trust code cannot control which environment variables are checked and cannot get the values of the environment variables. In order to do this successfully in a low-trust context, you will have to assert environment permissions to read the environment variables.

An assert is an elevation of privilege. When framework code does an assert, it is going to perform an operation the caller may not normally have permission to do (the framework must already have the permission, however, as well as the permission to perform an assert). Whenever framework code does an assert, that code must be audited carefully to make sure the assert is safe. This typically means checking the parameters passed in to make sure they've been validated and canonicalized where appropriate, making sure no inappropriate data is leaked back to low-trust code, and checking that low-trust code cannot inappropriately alter the state of high-trust code to trick it into doing something unsafe later on (known as a luring attack). Asserts are usually pretty easy to spot. You can search your source code for them, and there are a few FxCop rules related to asserts.

A more subtle and therefore potentially more dangerous elevation of privilege is satisfying a link demand. When a method has a link demand, its immediate caller is checked when the method is just-in-time (JIT) compiled. So, in effect, it is a one-level demand. It can be dangerous in that trusted code can satisfy a link demand, then turn around and expose that functionality to low-trust code without realizing it satisfied a link demand. One strategy trusted code might try in order to ensure security is to avoid doing any asserts and just let all demands flow to callers. But with link demands, trusted code is doing an implicit assert. To be fully secure, the trusted code would have to convert all link demands to full demands, as well. While there are some FxCop rules that help find code that satisfies link demands, you can't visually audit code to see if it satisfies any link demands. This makes auditing code more error prone.

Security-Transparent Code

Transparency, a new feature in the .NET Framework 2.0, is designed to help developers of frameworks write more secure libraries that expose functionality to low-trust code. You can mark an entire assembly, just some classes in an assembly, or just some methods in a class as security-transparent. Security-transparent code cannot elevate privilege. This has three specific ramifications:

  • Security-transparent code cannot perform asserts
  • Any link demand that would be satisfied by security-transparent code turns into a full demand
  • Any unsafe (unverifiable) code that must execute in security-transparent code causes a full demand for the skip verification security permission

These rules are enforced during execution by the CLR. Basically, security-transparent code passes all of the security requirements of the code it calls up to its callers. Demands just flow through it, and it cannot elevate privilege. So if a low-trust app calls some security-transparent code that causes a demand for high privilege, the demand will flow up to the low-trust code and fail—the security-transparent code could not stop the demand even if it wanted to. The same security-transparent code called from full-trust code will result in a successful demand.

There are three attributes used in transparency, summarized in Figure 6. When using transparency, you factor your code into security-transparent and security-critical (the opposite of security-transparent) methods. The bulk of your code that handles data manipulation and logic can usually be marked as security-transparent, while the small amount of your code that actually performs the elevations of privilege will be marked as critical. So far, groups within Microsoft that have adopted transparency have been able to mark upwards of 80 percent of their code as security-transparent. This has allowed them to focus auditing and testing efforts on the 20 percent of their code that is security-critical. For backward compatibility, all code in the .NET Framework 2.0 and previous versions that is not annotated with transparency attributes is considered to be security-critical. For now, you have to opt in to transparency. There are also FxCop rules for transparency, to help developers make sure they get the rules of transparency right when initially building their code, rather than having to debug a runtime error later.

Figure 6 Transparency Attributes (Close Figure 6)

Attribute | Description
SecurityTransparent | Only allowed at the assembly level. Opts the assembly into transparency and marks all types in the assembly as security-transparent. The assembly cannot contain any security-critical code.
SecurityCritical | When used at the assembly level, opts the assembly into transparency, and marks all code in the assembly as security-transparent by default, but signifies that the assembly may contain security-critical code too. When used at the class or method level, marks the class or method as security-critical. That class or method can now perform elevations of privilege.
SecurityTreatAsSafe | Can be placed on private or internal security-critical members to allow security-transparent code within the assembly to access the members. Otherwise, security-transparent code cannot access private or internal security-critical members within their assembly, as doing so may influence security-critical code and make unexpected elevations of privilege possible.

The reason for the SecurityTreatAsSafe attribute may not be obvious at first. Think of the security-transparent and security-critical code within your assembly as actually separated into two assemblies. The security-transparent code would not be able to see the private or internal members of the security-critical code. Additionally, the security-critical code is generally audited for access to its public interface. You would not expect private or internal state to be accessible outside of the assembly—you would want to keep the state isolated. So to allow the isolation of state between security-transparent and security-critical code while still providing the ability to override when necessary, the SecurityTreatAsSafe attribute was introduced. Security-transparent code cannot access private or internal members of security-critical code unless those members have been marked with SecurityTreatAsSafe. Before adding SecurityTreatAsSafe, the author of critical code should audit that member as though it were being exposed publicly.

Using Transparency

I'll use the math library example to demonstrate the syntax for transparency. First, to opt in to transparency, you have to add the SecurityCritical attribute as an assembly-level attribute:

using System.Security;

// opt in to low-trust callers
[assembly: AllowPartiallyTrustedCallers]
// opt in to transparency, but some code will be critical
[assembly: SecurityCritical]
Now all types in the assembly will be security-transparent by default. But let's say the constructor for the math library class needs to assert to read an environment variable, while all the other methods in the library can be security-transparent. The class definition would look something like Figure 7.
What Else is New in CAS?

Transparency and the simple sandboxing model for hosts are two of the biggest investments in CAS in the .NET Framework 2.0. But there are some other changes in CAS you should be aware of, as well.

The GAC is Full Trust

Assemblies in the GAC now always get full trust. This cannot be changed in policy or overridden by an AppDomainManager. Assemblies in the GAC are part of the trusted platform—you need admin rights to install an assembly in the GAC. Platform assemblies generally must run with full trust to ensure the proper functioning of the system. Trying to run them in a sandbox or reduce their privileges does not make sense and can lead to platform instability. The .NET Framework 1.0 and 1.1 provided the full-trust list, which had to include any assembly participating in policy, so most GAC assemblies were already getting full trust regardless of policy. Rather than having to know about both the full-trust list and the GAC, a framework developer only has to install their framework in the GAC now.

Identity Permission Demands are Satisfied by Full Trust

In the .NET Framework 1.0 and 1.1, you could use an identity demand, for instance for a StrongNameIdentityPermission, to create the appearance of being able to restrict the callers of your code based on identity. These demands would fail even if the calling code had full trust. The problem with this feature is that it is trivial to circumvent. A full-trust caller can explicitly load one of its assemblies with the strong name evidence of your strong name and then call your supposedly protected assembly. Given that this feature offers no security against full-trust callers, the behavior was changed so that full-trust callers will satisfy identity demands. If you are looking for some way to have "semi-internal" APIs that are shared across libraries your code controls, Take a look at the new InternalsVisibleTo attribute. It provides this functionality with none of the security implications.

You Cannot Turn off CAS Permanently

In the .NET Framework 1.0 and 1.1, you could completely turn off CAS enforcement by executing CASPol.exe –s off at the command line. While this is handy for some testing scenarios, it was easy to forget you had turned off CAS and thus not remember to turn it back on. This would leave your machine vulnerable to a number of attacks—any low-trust code that ran on your machine would have no restrictions on what it could do. For .NET Framework 2.0, CASPol.exe –s off does not permanently turn off security. Instead, when you execute this command, CASPol.exe will pause and wait for you to press a key. While CASPol.exe is paused, security is off. As soon as you press a key and CASPol.exe finishes and exits, security is back on.

Figure 7 Security-Critical MathLibrary (Close Figure 7)
public class MathLibrary
{
   private string m_procArch;

   // the constructor is critical
   [SecurityCritical]
   public MathLibrary()
   {
      new EnvironmentPermission(EnvironmentPermissionAccess.Read,
         "PROCESSOR_ARCHITECTURE").Assert();
      m_procArch = Environment.GetEnvironmentVariable(
         "PROCESSOR_ARCHITECTURE");
      CodeAccessPermission.RevertAssert();
   }

   // the rest of these methods are transparent

   public float ComputeFutureValue(float presentValue,
      float interestRate, int termInMonths)
   {
      ...
   }

   public int ComputeMedian(int[] values)
   {
      ...
   }
}

Transparency allows you to build frameworks in which most of the code runs in the context of its caller, while explicitly marking the code that can elevate privilege. This allows you to focus your security efforts on the code that is most sensitive security-wise, which is important as your codebase grows, and also makes code exposed to low-trust callers cheaper to build and maintain.

Conclusion

I've discussed the many possibilities for default sandboxes. Developers who want to host low-trust code or to extend the platform have to know more about CAS. New features have been added to the .NET Framework making it easier to host low-trust code and extend the platform more securely. As managed code spreads to more and more places, CAS continues to evolve, making it easier to sandbox low-trust code and build a more secure platform (for a look at other changes to CAS in the .NET Framework 2.0, see the sidebar "What Else is New in CAS?"). CAS for the .NET Framework 2.0 is designed to enable more developers to take advantage of these scenarios and enable more secure extensibility.

Mike Downen is the program manager for security on the CLR team. He works on Code Access Security, the cryptography classes, and the ClickOnce security model. You can read Mike's blog and contact him at blogs.msdn.com/CLRSecurity.

© 2007 Microsoft Corporation and CMP Media, LLC. All rights reserved; reproduction in part or in whole without permission is prohibited.


Do You Trust It?: Discover Techniques for Safely Hosting Untrusted Add-Ins with the .NET Framework 2.0

Do You Trust It?

Discover Techniques for Safely Hosting Untrusted Add-Ins with the .NET Framework 2.0

Shawn Farkas
This article is based on a prerelease version of the .NET Framework 2.0. All information herein is subject to change.
This article discusses:
  • Designing support for add-ins
  • Application domain sandboxing
  • New APIs for add-in support
  • Advanced add-in hosting
This article uses the following technologies:
.NET Framework 2.0
Contents
A Basic Add-In Model
Using Full Assembly Names
AppDomain Sandboxing
Changing the ApplicationBase
A Robust API for Add-Ins
Advanced Hosting Techniques
Conclusion

T
hrough the use of dynamic assembly loading and reflection, managed code provides a way for you to easily allow your application to be extended by add-ins. However, when you allow your application to run arbitrary code through an add-in model, you expose the user's computer to potentially unknown code, running the risk that malicious code will use your application as an entry point to the user's data. There are several techniques you can use to reduce the attack surface of your application and protect against malicious or otherwise unstable code corrupting your user's computer—or possibly stealing sensitive data.

In this article, I'll explore a hypothetical application that supports add-ins, and examine how the application might be augmented to provide additional features and functionality to the user while also helping to protect the user's computer from malicious or unstable code.

A Basic Add-In Model

In its simplest form, an application's add-in model is little more than an interface or set of interfaces defined in a shared library. Add-ins implement the interface, and notify the application of their existence, possibly by registering themselves in a configuration file or by placing themselves in a separate directory created specifically for add-ins.

In order to create one of these add-ins, the application would simply call Activator.CreateInstance, specifying the assembly containing the add-in and the type that implements the common interface. By casting the resulting instantiated object to the well-known interface, you can then invoke its functionality, as demonstrated in Figure 1.

Figure 1 Flawed Add-In (Close Figure 1)

In AddInManager.dll
public interface IAddIn
{
    void Start();
    ...
}
In Application.exe
IAddIn addIn = Activator.CreateInstance(
    "AddInAssembly",
    "AddInNamespace.AddInType") as IAddIn;
if(addIn != null)
{
    addIn.Start();
}

While this method will work, it has many potential pitfalls. For instance, by providing only a simple assembly name to Activator.CreateInstance, you know that you're loading an assembly named AddInAssembly, but you don't necessarily know whether you're loading the add-in that your user wants.

Using Full Assembly Names

Instead of using simple assembly names to load add-ins, you should always refer to them by their strong name. This prevents a scenario where a malicious assembly with the same name as a useful add-in is accidentally loaded by your application. Since the strong name of an assembly contains the public-key token that the assembly was signed with, you know that the assembly you're loading was created by someone with access to the private key of the add-in you intended to load.

You should also make sure to give strong names to all of the assemblies in your hosting application. By strongly naming your assemblies, you can ensure that add-ins bind to the version of your application that they were written against. If you need to release side-by-side versions of your app, then add-ins for version 1 of your host will bind against the version 1 add-in architecture, while add-ins for version 2 will bind against the version 2 architecture.

The simple add-in mechanism shown in Figure 1 has an even more obvious flaw than using simple names, however. In the current model, you're simply passing control of your application over to untrusted code that could do anything.

Since the add-in probably resides in a DLL on the local computer, the default security policy is going to grant that code full trust. This means that even though the add-in implements the IAddIn interface and claims to want to work within your add-in architecture, nothing is preventing it from accessing the user's Microsoft® Money files, for example, and copying them up to a Web site inside of its Start method.

Even if the add-in isn't explicitly malicious, it could be poorly coded. If something inside the Start method causes the add-in to crash (say, by throwing an unhandled exception), it has the potential to bring down your entire application with it—possibly causing any changes the user made to her document to be lost.

AppDomain Sandboxing

The basic unit of isolation in the common language runtime (CLR) is the application domain, represented by the System.AppDomain class. It's common to describe an application domain, or app domain, as a sort of lightweight process because code executing inside one app domain does not affect code executing inside another. In fact, unless they go through remoting or unmanaged code, they can't even access each other.

Another interesting aspect of an app domain is that it can be used to control the permission sets granted to different assemblies. When an app domain is used to run an assembly with a well-known, safe set of permissions, that app domain is generally referred to as a sandbox.

In version 1.x of the CLR, setting up a sandboxed app domain took significant effort. You had to create an AppDomain policy level, create a series of code groups, and define the permission sets that would be granted to each one. Thankfully, the .NET Framework 2.0 introduces a simple sandboxing API in the form of a new overload of the static AppDomain.CreateDomain method.

The simple sandboxing API creates an AppDomain that has two distinct permission sets. Assemblies from the Global Assembly Cache (GAC) receive full trust, while every other assembly receives a set of permissions that you specify at AppDomain creation time. The AppDomain itself is also configured with the specified permission set. Finally, you can specify a set of assemblies that do not reside in the GAC, but that should be granted full trust anyway. The API takes the five parameters described in Figure 2.

Figure 2 Sandboxing Parameters (Close Figure 2)

Parameter | Type | Use
friendlyName | string | Name of the AppDomain
securityInfo | Evidence | Evidence for the AppDomain
info | AppDomainSetup | Extra configuration settings for the AppDomain
grantSet | PermissionSet | Default permission set to grant assemblies and the domain itself
fullTrustAssemblies | StrongName[] | Assemblies to receive FullTrust in addition to all assemblies from the GAC

Note that the evidence used for the domain will never be used for code access security (CAS) policy evaluation. In fact, assemblies loaded into this type of AppDomain are never resolved against the security policy, since you've already specified the permission set that they should be granted. This means that even if you modify your machine policy such that it would grant extra permissions to an add-in you trust, when the add-in is loaded into an AppDomain created with the sandboxing API, the add-in won't receive those permissions.

A basic sandboxed application domain that grants the Internet permission set to code loaded into it might look something like the code in Figure 3. If you attempt to use that code, however, you'll notice a quirk in the parameters for CreateDomain. As is, this code will throw an InvalidOperationException, saying "This API requires the ApplicationBase to be specified explicitly in the AppDomainSetup parameter." The ApplicationBase property affects where the CLR looks for assemblies when attempting to load them. A quick fix for the exception would be to set the ApplicationBase to be the same as that of the default domain:

setup.ApplicationBase =
    AppDomain.CurrentDomain.SetupInformation.ApplicationBase;
By adding that bit of code, you ensure the sandboxed application domain will be created successfully.

Figure 3 AppDomain with Internet Permissions (Close Figure 3)
// get the set of permissions granted to code from the Internet.
Evidence internetEvidence = new Evidence(
    new object[] {  new Zone(SecurityZone.Internet) },
    new object[] {  });

PermissionSet internetPermissions =
    SecurityManager.ResolvePolicy(internetEvidence);

// create a sandboxed domain
AppDomainSetup setup = new AppDomainSetup();
AppDomain sandboxedDomain = AppDomain.CreateDomain(
    "Add-In Domain", internetEvidence, setup,
    internetPermissions, new StrongName[] { });

Now you'll need to load the add-in into that domain, and let it do its work. One way to do this is to define an add-in manager class that gets loaded into the sandbox and invokes the add-in for you (see Figure 4). This class should derive from MarshalByRefObject so that it can take advantage of remoting to communicate between the host AppDomain and the sandboxed domain (see Figure 5). This is a big improvement over the previous model, since now all add-ins are limited to the Internet permission set, and a crash in one add-in will not crash the entire application.

Figure 4 Creating an Add-In Manager (Close Figure 4)

Invoking the Add-In in the Sandbox
Type addInManagerType = typeof(AddInManager);
AddInManager manager = (AddInManager)
    sandboxedDomain.CreateInstanceAndUnwrap(
        addInManagerType.Assembly.FullName,
        addInManagerType.FullName);
manager.RunAddIn(addInAsembly, addInType, currentDocument);
In AddInManager.dll
public class AddInManager : MarshalByRefObject
{
    public void RunAddIn(string addInAssembly, string addInType)
    {
        IAddIn addIn = Activator.CreateInstance(addInAssembly,
            addInType) as IAddIn;
        if(addIn != null) addIn.Start();
    }
}

While the basic AddInManager might work, doing anything but the most basic tasks will probably cause a SecurityException since every assembly in the sandboxed app domain will be granted the Internet permission set unless it's located in the GAC. To grant FullTrust permissions to AddInManager, you'll need to add its strong name to the FullTrust list.

Figure 5 Communicating Across Domains
Figure 5 Communicating Across Domains

The CLR does not provide an easy method for getting a StrongName object for an assembly, although the code to do so is relatively straightforward, as shown in Figure 6. By adding the strong name of AddInManager.dll to the FullTrust list when creating the sandboxed domain, you ensure that the code will be free to do what it needs to while still limiting the add-in's permission set.

Figure 6 Finding the Strong Name for an Assembly (Close Figure 6)
public static StrongName GetStrongName(Assembly assembly)
{
    if(assembly == null)
        throw new ArgumentNullException("assembly");

    AssemblyName assemblyName = assembly.GetName();

    // get the public key blob
    byte[] publicKey = assemblyName.GetPublicKey();
    if(publicKey == null || publicKey.Length == 0)
        throw new InvalidOperationException(
            String.Format("{0} is not strongly named",
            assembly));

    StrongNamePublicKeyBlob keyBlob =
        new StrongNamePublicKeyBlob(publicKey);

    // create the StrongName
    return new StrongName(
        keyBlob, assemblyName.Name, assemblyName.Version);
}

Keep in mind that the sandboxed application domain itself has the restricted permission set. This means any security demand that causes a stack walk to attempt to leave the sandboxed domain will have to be a subset of the sandboxed permission set. In order to prevent this from hampering your add-in code, you'll need to assert FullTrust before doing any privileged operations.

When asserting FullTrust, you should always remember to revert the assert as soon as the privileged operation is complete. For instance, if you needed to read some configuration information out of a database in your add-in manager, the code might look like Figure 7. Remember that any time you assert permissions, make sure that you're not vulnerable to a luring attack where untrusted code can affect what you do while operating with full trust.

Figure 7 Asserting Permissions (Close Figure 7)
public class AddInManager : MarshalByRefObject
{
    private static void ReadConfiguration()
    {
        ... // prepare to connect to the database here

        new PermissionSet(PermissionState.Unrestricted).Assert();
        ... // read configuration information from the database
        CodeAccessPermission.RevertAssert();

        ... // process configuration information here
    }
    ...
}

Changing the ApplicationBase

Recall that my first attempt to create a sandboxed domain (see Figure 3) caused an InvalidOperationException because I had not set the ApplicationBase for the application domain. Since setting the ApplicationBase to be the same as the hosting application's ApplicationBase can have unintended consequences, the CLR needs you to explicitly set this property rather than using the ApplicationBase of the host AppDomain as the default.

The problem with using the same ApplicationBase as the host is that a partial trust assembly can use the Assembly.Load method to load any assembly that resides in the ApplicationBase. Once the assembly is loaded, the partial-trust code can then access any public types or methods defined in it. Generally, the hosting application's classes will not be intended to be used in this fashion. And it almost certainly won't have been tested in these conditions.

In order to prevent this repurposing attack, you need to prevent the partial-trust code from loading the host assemblies. The best way to accomplish this goal is to ensure that the add-ins are physically located in a different directory than the hosting application. Then, when creating the sandboxed domain, set the ApplicationBase to point to the add-in directory (see Figure 8).

Figure 8 Preventing a Repurposing Attack
Figure 8 Preventing a Repurposing Attack

There are two common schemes for organizing the layout of an add-in. The first is to place all add-in assemblies in a subdirectory of the hosting application (for example, \AppDir\AddIns). Another scheme is to have a separate subdirectory for each add-in. By separating the add-ins from each other, you prevent them from being able to load each other at run time.

Deciding between these two schemes is largely dependent upon the goals of your application. Separate directories for add-ins allow you to isolate each add-in from the others, but comes at the cost of requiring a new AppDomain for each add-in. On the other hand, placing every add-in in the same directory allows you to use only a single sandboxed AppDomain, but leaves the add-ins free to interfere with each other.

The first problem that you'll run into when moving add-ins to their own ApplicationBase is that the very issue you were trying to solve now comes back to bite you. Since the sandboxed AppDomain can no longer access host assemblies, there is no way for your AddInManager.dll to be loaded. There are three obvious solutions to this problem:

  • Have several copies of AddInManager.dll, one in your application's directory and one in each add-in directory.
  • Place AddInManager.dll in the GAC.
  • Use LoadFrom to direct the loader to the exact path of your AddInManager.dll.

Clearly the first method doesn't scale very well, especially if you choose to have a separate subdirectory for each add-in. In addition to wasting disk space, you now have a servicing nightmare if you need to deploy a security update to your AddInManager library.

By placing AddInManager in the GAC, you can avoid using LoadFrom. This can be desirable since in some situations LoadFrom can cause problems that are difficult to debug because it uses a different loading context than Load. However, since the assembly may need to be used by partially trusted code, you'll probably have to apply the AllowPartiallyTrustedCallers attribute. Now that it's in the GAC, any managed code that runs on the machine can access the assembly, so you'll need to do a security review and test your code for the cases in which it's not running as part of your app.

Using LoadFrom presents quite a different problem. LoadFrom requires FileIOPermission for the assembly being loaded. The sandboxed AppDomain won't have this permission—otherwise someone could just LoadFrom your host assemblies, and you would not have solved anything by changing the ApplicationBase. Calling the standard AppDomain.CreateInstanceFrom method will cause a SecurityException when the demand hits the AppDomain boundary.

In order to solve this problem, the .NET Framework 2.0 introduces a new overload of Activator.CreateInstanceFrom. This overload takes an AppDomain to create the instance in, the path to the assembly, and the type to create. This overload is special because, before loading the assembly, it does an assert for full trust in order to prevent the demand for FileIOPermission from failing. This method would be used like this:

// create an add-in manager to invoke the add-in in the sandbox
Type addInManagerType = typeof(AddInManager);
AddInManager manager = (AddInManager)Activator.CreateInstanceFrom(
    sandboxedDomain,
    addInManagerType.Assembly.Location,
    addInManagerType.FullName).Unwrap();
manager.RunAddIn(addInAsembly, addInType);

There's also a corresponding version of CreateInstance that behaves in a similar way. If you need to pass parameters to the constructor of your object, or want to affect the way the runtime binds to your type, there are other overloads that provide such functionality. The rule of thumb is that any method on Activator that takes an AppDomain as a parameter is going to operate under an assert for full trust.

While these methods can be convenient, they can also be dangerous if not used carefully. When these methods are used, the constructor of the type being instantiated will run with no partial- trust boundary on the call stack. This has several implications that your application needs to be hardened against.

First, you should not pass any untrusted objects as parameters to your constructor. These objects could implement custom serialization, which would be invoked when they were transferred across the AppDomain boundary. Since there's no way for the CLR to ensure that the object's serialization code is actually doing the work of serialization, passing untrusted objects as parameters gives these objects a window to do just about anything they want. Since there's nothing but FullTrust frames on the call stack, the serialization code is free to upload your user's Microsoft Money files (or any other files) to the Internet again.

Another related issue has to do with exceptions. The constructor for your AddInManager object needs to make sure no untrusted exceptions leave its stack frame. Since the CLR will attempt to move the exception from the sandboxed application domain back into your host domain, the exception could use its serialization code to run with full trust.

Not allowing any untrusted exceptions out of your constructor means that you need to have a blanket try/catch around any calls to partial-trust code. This includes not only direct calls to methods in untrusted assemblies, but also calls to virtual methods that could have been overridden by untrusted code and calls that will end up invoking delegates that could come from the untrusted code. In general, it's safest to do as little work as possible in your constructor and to avoid calling any untrusted code.

A Robust API for Add-Ins

In all but the simplest applications, you'll want to provide an object model with which your add-ins can interact. This object model will probably consist of at least one System.Security.AllowPartiallyTrustedCallersAttribute (APTCA) assembly, which exposes several classes the add-ins can use to interact with your application and its open documents. Since this assembly will be marked with the APTCA attribute, security reviews are vital. If your object model grows to any significant size, this can be time-consuming, but absolutely necessary to make sure your app is not exposing a path for untrusted code to elevate its permissions.

Thankfully, the .NET Framework 2.0 provides transparent assemblies that help to reduce the attack surface and the amount of code that must be thoroughly audited. A transparent assembly voluntarily gives up its ability to use constructs such as unverifiable code and asserts. Transparent code also cannot satisfy a LinkDemand. Due to these restrictions, CAS demands just propagate through the transparent code, even if it is a FullTrust assembly. However, a partial-trust transparent assembly may still cause a demand to fail. Since a transparent assembly cannot be used by partially trusted code to increase its effective permission set, any assemblies that are marked transparent do not require as thorough of a security audit as standard APTCA assemblies.

In order to reduce the attack surface of your application, one good way to structure your object model would be to create most of it in a transparent assembly. Then some security-critical code can be used to broker access from the transparent assembly to the rest of the application. This minimizes the amount of security-critical code that needs review.

Assemblies are marked transparent with the SecurityTransparentAttribute. Once the assembly has been marked transparent, no security-critical code can reside within it. If you need to mix security-critical and transparent code in the same assembly, you should instead apply the SecurityCriticalAttribute to the assembly, which makes the assembly critical, but does not automatically apply to all code within the assembly. Instead, you must mark all critical code with its own SecurityCriticalAttribute. This makes everything in the assembly not specifically marked with a SecurityCriticalAttribute transparent, but gives you the flexibility to mark specific types and methods as security-critical.

If you do want to have the SecurityCriticalAttribute propagate to everything scoped beneath its target, you should pass it a parameter of SecurityCriticalScope.Everything. For instance, if you marked a class with SecurityCriticalScope.Everything, all nested types, methods, and properties within the class would also be marked as security-critical.

As an example of designing with a split between transparent and critical code, let's consider the object model that a text editor might expose. The editor could have an Application class with a CurrentDocument property exposing the document the user is currently working on. Once obtained, the Document class lets the add-in obtain its size, which doesn't require any special permissions. However, it also allows the add-in to attempt saving the document. Since saving will require access to the file system, it will require elevated permissions. The resulting object model might look, in part, like Figure 9.

Figure 9 Application Class (Close Figure 9)
[assembly: AllowPartiallyTrustedCallers]
[assembly: SecurityCritical]

public sealed class Application {
    public static Application Instance { get { return currentInstance; } }
    public Document CurrentDocument { get { return currentDocument; } }
    ...
}

public sealed class Document {
    public int Size { get { return contents.Length; } }

    [SecurityCritical]
    public void Save()
    {
        if(EnsureUserWantsToSave()) {
            new FileIOPermission(PermissionState.Unrestricted).Assert();
            WriteDocumentToDisk();
            CodeAccessPermission.RevertAssert();
        }
    }
    ...
}

In this case, since the assembly contains some critical code, the entire assembly requires a SecurityCriticalAttribute instead of a SecurityTransparentAttribute. However, since Application doesn't do any security actions, it doesn't get a SecurityCritcalAttribute and is transparent. If you attempted to do an Assert in one of Application's methods, you would get an InvalidOperationException with a message of "Cannot perform CAS Asserts in Security Transparent assemblies." The message is slightly misleading, since the assembly itself is not transparent, but the issue is that all code within the Application class is transparent.

Since the Save method of the Document class requires an Assert for FileIOPermission, that method must be marked critical. Within the method, the window of time that the Assert is in effect is kept as short as possible.

Advanced Hosting Techniques

The .NET Framework 2.0 provides the ability for applications to hook and customize many important CLR events, such as the creation of AppDomains and the resolution of security policy. The entry point to these features is the AppDomainManager class, which can be subclassed by applications wishing to customize the way the CLR handles these events. The AppDomainManager provides many methods and properties that can be overridden depending on the behavior you want to customize. A good overview of the options available can be found in Steven Pratschner's book, Customizing the Microsoft .NET Framework Common Language Runtime (Microsoft Press®, 2005)

For the purposes of add-in management, one of the more interesting events that you can hook is CreateDomain. By intercepting every call to CreateDomain, you can create a custom domain allocation policy. For instance, instead of having only one AppDomain for all add-ins to run in, you could put every add-in signed with the same key into the same AppDomain.

In order to implement this, you could attach the strong name of the add-in assembly that the AppDomain is for to the AppDomain's evidence and then call CreateDomain. Your AppDomainManager would then override the CreateDomain method and determine whether this public key already has an AppDomain or a new one should be created. Figure 10 shows how this could be implemented.

Figure 10 Custom Domain Allocation (Close Figure 10)
// create Evidence for the add-in being loaded
Assembly addInAssembly = Assembly.ReflectionOnlyLoadFrom(addInPath);
Evidence addinEvidence = new Evidence(
    new object[] { GetStrongName(addInAssembly) },
    new object[] {  });

// create a sandboxed domain
AppDomain sandboxedDomain = AppDomain.CreateDomain("Add-In Domain",
    addinEvidence, setup, internetPermissions, hostAssemblies);

public sealed class AddInAppDomainManager : AppDomainManager
{
    private Dictionary<StrongNamePublicKeyBlob, AppDomain> appDomains =
        new Dictionary<StrongNamePublicKeyBlob, AppDomain>();

    public override AppDomain CreateDomain(string friendlyName,
        Evidence securityInfo, AppDomainSetup appDomainInfo)
    {
        // find the strong name evidence
        StrongName strongName = null;
        foreach(object evidence in securityInfo)
        {
            strongName = evidence as StrongName;
            if(strongName != null) break;
        }

        if(strongName == null)
            throw new InvalidOperationException("Attempt to create " +
                "an AppDomain without add-in StrongName evidence");

        // see if we already have an AppDomain for this key
        if(appDomains.ContainsKey(strongName.PublicKey))
            return appDomains[strongName.PublicKey];

        // if not, create a new one
        AppDomain newDomain = CreateDomainHelper(
            friendlyName, securityInfo, appDomainInfo);
        appDomains[strongName.PublicKey] = newDomain;
        return newDomain;
    }
}

There are a couple of things to notice about this code. First, when loading the add-in assembly to get its strong name, the code only loads it for Reflection. The ReflectionOnlyLoadFrom method was added to the .NET Framework 2.0 to provide a way to inspect an assembly while being assured that no code from the assembly will be able to run.

Second, since AppDomainManager itself is managed code, it requires an AppDomain in which to execute. This means CreateDomain won't be called for the default domain, which is a good thing because you don't want to use your customization there.

Now that you've got an AppDomainManager, you need tell the CLR to load it. The easiest way to do this is to set the APPDOMAIN_MANAGER_ASM environment variable to be the full strong name of the assembly containing the AppDomainManager, and to set the APPDOMAIN_MANAGER_TYPE environment variable to be the namespace and type that derives from AppDomainManager. Note that the CLR requires AppDomainManger to be fully trusted and strongly named.

Conclusion

The ability to easily run add-in code within your app is one of the more powerful features of the .NET Framework. It can also be one of the more dangerous features if not used properly.

Always run untrusted code in a sandboxed AppDomain, with an ApplicationBase that is not the same as the ApplicationBase of your main application. If there is no partial-trust stack frame currently on the call stack, make sure that you do not allow custom serializers to run by passing untrusted objects across AppDomain boundaries or allowing untrusted exceptions to be marshaled across AppDomains. When providing an object model for the add-in code to program against, try to keep as much of it as possible transparent, only using security-critical code where absolutely necessary. When you follow these guidelines, you can provide a rich extensibility story for your applications, while keeping your user's data safe from being damaged by malicious code.

Shawn Farkas is a software design engineer in test on the CLR security team at Microsoft. He works on ClickOnce, cryptography, and IL verification. See his blog at blogs.msdn.com/shawnfa.


Are You Protected?: Design and Deploy Secure Web Apps with ASP.NET 2.0 and IIS 6.0

Are You Protected?

Design and Deploy Secure Web Apps with ASP.NET 2.0 and IIS 6.0

Michael Volodarsky
Parts of this article are based on a prerelease version of ASP.NET 2.0. Those sections are subject to change.
This article discusses:
  • Protecting resources and access control
  • Authentication and authorization
  • Code access security in ASP.NET
  • Application auditing
This article uses the following technologies:
ASP.NET, IIS 6.0, SQL Server
Contents
Designing Secure Web Applications
Protecting Resources
Access Control
Authentication
Authorization
Granular Application Authorization
Resource Access
Client Ticket Security
Preventing Replay
Preventing Attacks Through Input Validation
ASP.NET and CAS
Auditing with Web Events
Wrapping It Up

W
eb applications are among the most common computing services that are exposed to the Internet, and thus they pose an inviting target to anyone who wants to break into your network to steal sensitive information, tamper with your data, or otherwise compromise your system. Ensuring the security of a Web application is a serious task, and requires consideration throughout the design, development, deployment, and operation phases. It should not be viewed as something that can be slapped onto an existing application, or achieved simply by applying existing platform security features.

Even when developed for use within a relatively secure platform, a Web application must follow the security best practices of the platform throughout the design, development, and deployment stages to provide the maximum level of protection against attack. In order to produce secure Web app, this platform-specific knowledge should be combined with secure design practices, threat modeling analysis, and security penetration testing.

This article discusses best practices that allow you to take advantage of the security features of ASP.NET 2.0 and IIS 6.0 to build and deploy more secure Web applications.

Designing Secure Web Applications

Secure design principles usually boil down to a few key principles: assume all input into the system is malicious, reduce the exposed surface area of the system, lock down the system by default, and use defense-in-depth rather than relying on other parts of the system for protection. Following these general principles from the design phase onward is key to making sure your application is as well protected as possible.

Threat modeling analysis is used to map out the data flow inside a system and to examine the possible entry points a malicious user may be able to exploit for access to the system. Threat modeling is a key exercise for changing your perspective from that of a designer to that of an attacker, helping you to discover the potential security holes in your application. For more information on threat modeling, see Threat Modeling.

Security penetration testing is another name for hacking your own application—attempting to directly compromise your application to find problems before others do it for you. You can try pushing the application to its limits by sending invalid or worst-case data. You can also try foiling your application by using your knowledge of its internal workings—in this case, your insider knowledge gives you an unfair advantage over typical attackers, but may also unearth vulnerabilities that were otherwise buried deep within your code. There are a variety of tools available that can help you do penetration testing.

Protecting Resources

Usually, the first task of deploying a Web application is making sure that it does not expose sensitive resources that can be accessed by anonymous clients on the Internet. If this is an intranet application, this is usually accomplished by making sure that all clients are authenticated with Windows® authentication (more on access control later), and that the application is protected from external access by a firewall. For an Internet application, this is more of an issue because it almost always needs to be at least minimally accessible by anonymous Internet users. Aside from this difference, the following guidance is nonetheless equally applicable to protecting resources for both Internet and intranet applications.

Ideally, it is best to make sure that your application's Web namespace does not contain any files that are not intended to be served to the client. This means removing all such files from the physical directory structure starting with the topmost directory marked as a Web application or virtual directory in the IIS configuration. If the file is not in the Web namespace, it should not be accessible by making requests to that namespace, unless your application code takes direct steps to open the file and serve its contents. If your application is programmatically accessing data or supporting files during its execution, you should place them outside the Web namespace.

If you cannot remove all such files, make sure that IIS is configured not to serve these files to Web clients. This should be accomplished by mapping the extensions for the protected files to ASP.NET in the IIS script processor map configuration (see Figure 1), and subsequently mapping those extensions to the HttpForbiddenHandler in the ASP.NET <httpHandlers> configuration for the application or directory. By default, ASP.NET already blocks access to a number of extensions common to Web applications, including .cs, .java, .mdb, .mdf, .vb, and others.

Figure 1 Mapping XML Files to the ASP.NET ISAPI Extension
Figure 1 Mapping XML Files to the ASP.NET ISAPI Extension

Configuring the Web.config as follows prohibits the .xml extension from being served if the extension has been mapped to ASP.NET for this application:

<configuration>
    <system.web>
        <httpHandlers>
            <add path="*.xml" verb="*"
             type="System.Web.HttpForbiddenHandler" />
        </httpHandlers>
    </system.web>
</configuration>
Note that this configuration generates a "403 Forbidden" response with an ASP.NET error message that gives away the existence of this file. You can instead use the System.Web.HttpNotFoundHandler in order to mask the existence of the file and return a generic 404 response.

If you uninstall ASP.NET, the IIS script processor maps for it will be removed and the unmapped extensions will be processed by the IIS static file handler. It is frequently thought that removing a corresponding extension entry from the IIS MIME mapping configuration will prevent the static file handler from serving files with that extension. In actuality this is not always true because the static file handler will also honor the registry MIME mapping configuration and therefore will still serve the extension if the registry entry is present. Because of this, you would have to make sure that both the metabase MIME mapping configuration and the registry key in HKEY_CLASSES_ROOT do not contain an extension entry for the forbidden extension. Due to the management complexity, this practice is not recommended as a way of securing content. If you do use this method, you should also make the files hidden for additional protection because the IIS static file handler will not serve files that have been given the hidden attribute.

Of course, this guidance regarding the IIS static file handler only applies to extensions that are not mapped to an ISAPI extension in the IIS script mapping configuration. If the extension is mapped, then the ISAPI extension in the corresponding IIS script mapping will be responsible for controlling access to requests that have that extension.

You can also take advantage of the protected directories feature in ASP.NET 2.0. By default, ASP.NET 2.0 will block access to all URLs that contain directory segments named App_Data, App_Code, App_Browsers, App_WebReferences, App_GlobalResources, and App_LocalResources anywhere in the URL. ASP.NET 1.1 and 2.0 will both also block access to the /Bin directory. This support is conditional on having the aspnet_filter.dll ISAPI filter registered with IIS, which is done by default during ASP.NET installation. By placing your content in these directories, you can prevent HTTP access to it regardless of the file extensions being requested. Figure 2 describes the protected directories under ASP.NET 2.0. URLs containing other App_* segments, including the App_Themes directory, are not blocked by ASP.NET.

Figure 2 Paths Denied Access via Direct HTTP Requests ASP.NET 2.0 (Close Figure 2)

Directory | Description
/Bin | Contains managed application assemblies. This is also blocked in ASP.NET 1.1.
/App_Code | Contains compileable application code.
/App_Data | Contains application data files.
/App_LocalResources | Contains directory-local resources.
/App_GlobalResources | Contains application-global resources.
/App_WebReferences | Contains Web references generated by Visual Web Developer.
/App_Browsers | Contains browser definitions.

Access Control

Most Web applications provide multiple levels of access to data and resources, and need to ensure that clients receive appropriate access based on their credentials. Access control in Web applications is typically handled in two stages: authentication and authorization. Authentication refers to the process of determining the identity of the client making the request. Authorization refers to the process of determining whether that client identity is allowed to access a particular resource. Figure 3 illustrates the security-relevant stages of request processing of a typical HTTP request on the IIS/ASP.NET platform.

Figure 3 Processing an HTTP Request
Figure 3 Processing an HTTP Request

There are a number of non-user access control mechanisms available that you may want to consider for your application to reduce its attack surface. These mechanisms on IIS 6.0 include the IP restriction list and the Web service extension restriction list.

The IP restriction list allows you to specify the client IP addresses that are allowed access into your application. You can configure specific IP addresses and domains as well as subnets. Consider doing this if your application is intranet only, or if your clients always access your application from a specific IP address or domain. When applicable, this should be used as a defense-in-depth measure in conjunction with other access control methods.

The extension restriction list allows you to globally define the ISAPI extension DLL files and the CGI .exe files that can be executed on the server. By default, all such extensions are prohibited, and you should enable only the ones that you need and leave the other ones disabled. Third-party product installations may enable additional components, so you might need to disable them if you are not using them to serve Web content.

Previous versions of IIS also benefited from the UrlScan ISAPI filter, which provided additional request validation and blocking of dangerous input. IIS 6.0 has been reengineered to provide greater security than previous versions, and no longer exposes vulnerabilities that required UrlScan for protection. However, if your security requirements are not covered by the features provided in IIS 6.0 by default, you may want to continue using UrlScan. The information available at Determining Whether to Use UrlScan 2.5 with IIS 6.0 can help you decide whether to use UrlScan.

Authentication

IIS and ASP.NET together provide a foundation for building access control mechanisms, and ASP.NET 2.0 further expands upon this to provide ready-to-go application blocks that can be used to deploy such mechanisms quickly.

The output of IIS authentication mechanisms is always a Windows identity that represents the user from whom the request is being made. IIS provides built-in support for the following authentication types: anonymous, Integrated Windows, Basic, Digest, certificate mapping, and Microsoft® Passport. Figure 4 illustrates when different authentication modes are typically used.

Figure 4 Authentication Methods (Close Figure 4)

Authentication Method | Application Type | Users Must Have Windows Accounts | Comments
Anonymous | Both | No | Used with application-level authentication schemes.
Integrated Windows | Intranet | Yes | Internet usage is possible but not recommended.
Basic | Internet, or intranet when firewall or domain trust limitations prohibit using Integrated Windows authentication. | Yes | Password sent in clear text. Must use SSL to prevent password exposure.
Digest | Internet | Yes | Requires Active Directory.
Certificate mapping | Internet | Yes | Requires a client certificate to be issued.
Passport | Internet | No, but must have Passport account. | Requires Passport affiliation.

Intranet applications typically will use the Integrated Windows authentication to authenticate clients with their domain accounts; alternatively, they may use certificate authentication, though certificates are more common in the Internet realm. Internet applications should use Basic authentication over SSL or Digest authentication if users have Windows accounts, or anonymous authentication with an application-specific authentication scheme if they do not have Windows accounts.

If anonymous authentication is not enabled for a URL, then IIS will either make sure that the request is authenticated or it will reject the request. This should be used as a defense-in-depth measure for applications using other IIS authentication mechanisms (most intranet applications) to outright reject all unauthenticated users if no anonymous access is desired.

ASP.NET supports three types of authentication: Windows, Forms, and Passport. Windows authentication simply accepts the Windows identity produced during IIS authentication. Forms authentication implements a ticket-based application layer authentication scheme that should be used by ASP.NET applications that do not associate Windows accounts with their users. (Forms authentication should be combined with the IIS anonymous authentication option.) Passport authentication uses the Microsoft Passport authentication scheme. The authentication scheme for each application is specified in the <system.web><authentication> configuration element, as shown here:

<configuration>
    <system.web>
        <authentication mode="Forms" />
    </system.web>
</configuration>
With forms authentication, when the user is denied access for a resource that requires authentication, the user is then redirected to a configured URL that points to a login page. The login page is a custom ASP.NET page written by the application developer, which provides a way for the user to supply credentials. The login page then validates the credentials against a custom user store, such as a SQL database, and uses the System.Web.FormsAuthentication class to issue an encrypted authentication ticket to the client. This ticket can be either a cookie or a special blob that augments the URL (starting with the ASP.NET 1.1 Mobile Toolkit and ASP.NET 2.0). On every subsequent request, the client presents this ticket, and the forms authentication mechanism automatically processes it to generate the user identity for the request.

With forms authentication, the complexity of issuing and managing the authentication ticket is absorbed by the ASP.NET framework. However, you still have to write the login page and the associated user store implementation. ASP.NET 2.0 simplifies the entire process by providing two additional components: the Membership service and the Login control family.

The Membership service, exposed by the System.Web.Security.Membership class, provides a useful abstraction for creating, managing, and validating user credentials. Like many other ASP.NET 2.0 features, the provider-based design enables the service to work with a variety of back-end data stores, including SQL Server™ and Active Directory® providers that are included in the ASP.NET framework. Setting up your own user database implementation is as easy as deploying the Membership database schema to your SQL Server or SQL Server Express instance through the aspnet_regsql.exe tool, and configuring a SqlMembershipProvider—or just using the default provider that automatically creates the SQL Server Express membership database in your application's App_Data directory on first access.

The Membership service provides support for many common user management functions, including password hashing, account lockout, and login tracking. You can learn more about the Membership service in the ASP.NET 2.0 Quickstart (Using the Membership and Role Manager APIs), and in Dino Esposito and Andrea Saltarello's article in this issue of MSDN®Magazine.

With the ASP.NET 2.0 Login control, the crux of the login page creation is simplified even further by providing a drag and drop control that implements all of the login page functionality automatically by working against membership and forms authentication APIs to validate the user credentials and create the authentication ticket on successful logon. The Login control provides flexible functionality, including working with custom Membership providers, performing question/answer password reset, and doing custom credential validation. A set of supporting controls, including LoginView and CreateUser controls, can be used to support the rest of your user management interface (see Using the Login Controls).

When using custom application authentication with forms authentication or your own authentication mechanism, protect your login page and preferably your entire application with the secure sockets layer (SSL) to prevent disclosure of login credentials and authentication tickets. (I'll cover authentication ticket security in more detail later in this article.) Also, you should require your users to use strong passwords. The Membership service provides the ability to set regular expressions for enforcing password strength and format. You should make sure that hashed passwords are used (which is the default behavior), and you should not store credentials in the forms authentication <credentials> section in your application configuration file.

Authorization

After a user identity for the request has been established, the identity is stored in the request context as a System.Security.Principal.IPrincipal object, which is available from HttpContext.Current.User. This object can then be used to make access control decisions for the request.

ASP.NET provides two built-in authorization mechanisms that control access on the URL level: File authorization and URL authorization. Both of these execute after authentication, in the AuthorizeRequest ASP.NET pipeline stage, and check whether the requesting identity has access to the requested URL. If the identity is determined not to have access, the request is terminated with a "401 Unauthorized" response.

File authorization works only when the Windows authentication option is used (when the user principal contains a Windows identity), and uses file access control lists (ACLs) to determine whether the requesting identity should be allowed access to the URL. Note that File authorization also only works when the the request can be mapped to a file and the file is on a local machine. If the physical file is on a share, this authorization has no effect and allows the request.

File authorization can be used by applications that authenticate users who have Windows accounts and who use ACLs to control access to their resources. Be sure to remove read access for any user identity that should not have access to a given file when using this type of authorization.

URL authorization uses configuration-based access control rules that reference the user's name or roles as a way to grant or deny access. It is commonly the preferred authorization mechanism because it does not depend on the user identity being a Windows account, it can be used with any authentication mechanism that produces a user principal, and it offers an easier way to manage access policies than by managing ACLs on files.

URL authorization access control rules can be specified per-URL in the <system.web><authorization> configuration section. The authorization mechanism will process the rules in the top-down order and select the first rule that is matched, taking the deny or allow action specified by the rule. The following configuration allows all authenticated users and denies anonymous users access for the URL scope:

<authorization>
     <allow users="*" />
     <deny users="?" />
</authorization>

Alternatively, you can also qualify the rules with user names or roles. In ASP.NET 1.1, you have to manually associate a list of roles with a user principal after authentication. In ASP.NET 2.0, you can use the System.Web.Security.RoleManager class to create and manage roles and have these roles automatically associated with a user after authentication. You can also use location tags to specify access control rules for specific URLs in the same Web.config file.

URL authorization should be used only to allow access to the specific set of users, and to deny access to everyone else. A good practice involves designing a more restrictive configuration at the root directory level (deny everyone or deny all anonymous) with specific relaxation for individual URLs or subfolders that have less stringent access requirements. You should consider the authorization requirements of your application early in the design stage, and try to group URLs with similar access requirements together in directories with a single set of authorization rules to minimize the need to configure many individual rules, and to reduce the risk of getting them wrong.

Granular Application Authorization

URL-level authorization is a great way to secure your application, and should be used wherever possible to restrict access to the minimum set of users that need to interact with your system. However, sometimes further application-level authorization is required in order to provide more fine-grained control over the functionality that is available to your users.

Typically, application authorization is performed by inspecting the current authenticated user principal, available through the HttpContext.Current.User property, and making a decision to grant or deny access to the specific functionality based on either the user name or role membership of the principal. In ASP.NET applications, this is typically done in two stages: hiding page regions, and validating access programmatically before performing server actions.

Hiding page regions is a common practice in existing applications, and entails selectively showing a set of page controls by placing these controls in a panel that is set to be visible or invisible based on the user identity or role membership. In ASP.NET 2.0, this task is simplified via the LoginView control, which provides an easy way to associate templates with users and roles. (Take a look at Using the Login Controls for an example on how to use this control.) Note, however, that this practice alone is not enough.

In addition to hiding portions of a page from being visible, you need to enforce access control in the actual actions taken by your application in response to user input. These actions are most commonly ASP.NET page postbacks that invoke server event handlers, or code in your page event handlers that acts on a querystring, form data, or other client input. In ASP.NET 1.1, hiding a control does not prevent a malicious client from being able to trigger its event handlers, so this is a mandatory practice. In ASP.NET 2.0, the controls in the System.Web.UI.WebControls and System.Web.UI.HtmlControl namespaces prevent by default postback events from being raised if they are hidden, and custom controls can also do the same.

To enforce access control, place code in every event handler or method that performs a sensitive action to make sure the current user is authorized to perform the current action. This way, even if the client manages to directly invoke the event handler, you will ensure that only authorized users can perform the protected action. You can do this by directly checking the user principal, or by using declarative or imperative demands for the user principal.

The code shown in Figure 5 ensures that the CreateDatabase event handler only allows the Administrator user name or all users in the Operator role to complete the call. Alternatively, you can enforce code authorization by making demands against the System.Security.Permissions.PrincipalPermission class. This class can be used to make demands for a particular user name and role against the current user principal set on the thread. This can be used to do both programmatic and declarative demands against the user principal. Figure 6 shows two examples of using the PrincipalPermission to require the user to be Administrator.

Figure 6 Authenticating with PrincipalPermission (Close Figure 6)
[PrincipalPermission(SecurityAction.Demand,
                     Name="Administrator",
                     Role=null)]
void CreateDatabase_Click(Object source, EventArgs e)
{
    ...
}

void CreateDatabase_Click(Object source, EventArgs e)
{
    PrincipalPermission p = new PrincipalPermission(
        "Administrator", null);
    p.Demand();
    ...
}
Figure 5 Access Control for Event Handlers (Close Figure 5)
void CreateDatabase_Click(Object source, EventArgs e)
{
    System.Security.Principal.IPrincipal user = Context.User;

    if (user != null && user.Identity != null)
    {
        if (user.Identity.Name.Equals("Administrator") ||
            user.IsInRole("Operator"))
        {
            CreateDatabase();
            return;
        }
    }

    throw new System.Security.SecurityException(
        "You are not authorized to perform this action!");
}

ASP.NET will set the current request principal on the thread after all authentication modules and the global.asax AuthenticateRequest event handlers have executed. This means that if you want to set a custom principal on the request after this stage, you have to set the thread principal manually by assigning your principal object to the Thread.CurrentPrincipal property.

Resource Access

In addition to the user-based authentication and authorization I've discussed, it is also important to consider how Windows resources (files, registry, objects, remote SQL Servers) are accessed within an ASP.NET application. Because ASP.NET user abstraction is decoupled from Windows accounts, the identity used for accessing Windows resources is not always the identity of the authenticated user. In fact, the accessing identity depends on the Windows identity impersonated by an application thread, and can be any of the identities described in Figure 7.

Figure 7 Application Thread Identities (Close Figure 7)

Identity | Description | Configuration
Process Identity | The identity of the worker process that houses the ASP.NET application. This is typically Network Service in IIS 6.0 Worker Process Isolation mode (default), or ASPNET in IIS 5.0 Isolation Mode. ASP.NET will, by default, use this identity to access resources (except for UNC). | The IIS 6.0 application pool identity in WP mode, or the account specified in the <system.web><processModel> ASP.NET configuration section. No impersonation settings.
Authenticated User Identity | The identity of the authenticated user. This is used to access Windows resources with the client identity in intranet applications, such as for auditing purposes, to sandbox intranet users, or in order to manage access with Windows accounts. | IIS authentication method other than anonymous was used. If anonymous was used, the identity is the IUSR_MACHINE account by default or the configured anonymous account. ASP.NET is configured to impersonate the client with <system.web><identity impersonate="true" />.
UNC User Identity | The identity of the authenticated user, or the configured UNC user for IIS virtual directories located on a UNC share. | The URL maps to a UNC share, and IIS is configured for fixed account UNC authentication.
Fixed Application Identity | The identity specified in the ASP.NET configuration settings. This is typically used to access resources with a specific or lower-privileged account, or to connect to network resources with a delegated domain account. | ASP.NET is configured to impersonate a specific identity with <system.web><identity impersonate="true" username="account" password="password" />.

Intranet applications may chose to impersonate the client in order to limit the application to the rights of the accessing user. However, this also increases the burden of managing the resource permissions (file ACLs and SQL Server permissions, for example) for the application, and introduces performance overhead due to, among other reasons, inefficient database connection pooling. Additional limitations exist for remote resource access, such as accessing a remote SQL Server database with Integrated Authentication, because most IIS authentication schemes do not produce authentication tokens that can access remote machines on your network (except for Basic authentication, or if you have configured Windows NT® Lan Manager or Kerberos delegation).

An alternative is to use the trusted subsystem design, where your Web application accesses internal resources with a single identity, and ensures that its users have the permission to access these resources by doing application-level authorization (described previously). This is the approach taken by default by ASP.NET with the no-impersonation process identity mode. You can also achieve the same result with a fixed application identity, in which case you should be sure to protect the configuration credentials via configuration encryption.

When using the trusted subsystem approach, keep in mind the following best practices:

  • Perform early authorization (as described earlier in this article) and deny access to all except for the authorized users.
  • Run your application with the least privilege possible. If it's compromised, the attacker can only do as much damage as the process identity or application identity allows.
  • Only grant the minimal necessary access to resources. With SQL Server, for example, do not grant the application identity DataBase Owner (DBO) rights. Instead, only grant rights to the stored procedures necessary for its functionality, and withhold all other rights, including SELECTs on the tables.

Client Ticket Security

ASP.NET Web applications commonly use client tickets to associate state with a particular user. For example, the ASP.NET forms authentication mechanism described previously uses an encrypted client cookie to enable a client browser to access the application for a period of time without requiring the credentials to be provided by the user on every request. Other users of client tickets include Role Manager, which uses a cookie to associate a set of application roles with the client, and Session State, which uses a cookie to identify the session id for the client session. All of these features can also use URL-based tickets in ASP.NET 2.0, as I'll describe later. Application code can also store custom state in ViewState, form fields, and custom cookies.

Because the tickets are exposed to the client, they can be stolen, replayed, and altered by malicious clients, and that can result in a number of threats against the application. Specifically, these threats include malicious clients stealing a ticket from a legitimate user, and masquerading as that client by replaying that ticket or examining it for useful information about your application. Malicious clients could also craft a custom ticket to pretend to be another user or exploit the application in a specific way.

The following is general guidance that applies to all client tickets used by an application that can help mitigate or reduce the risk of all these threats. If the ticket contains secure information that you don't want the client to know, encrypt the ticket. This is supported by the forms authentication, Role Manager, and ViewState features. For forms authentication, set the <system.web><authentication><forms> protection attribute to All, which is the default setting. For Role Manager, set the <system.web><roleManager> cookieProtection attribute to All.

For ViewState, set the viewStateEncryptionMode attribute to Always in the @ Page directive or in the <system.web><pages> element. Note, however, that this functionality is available in ASP.NET 2.0 only. In ASP.NET 1.1, encryption can be achieved by using the 3DES ViewState validation.

The encryption performed depends on the settings in the <machineKey> element. Using encryption addresses the second threat, malicious users examining the ticket for useful information about your application. The tradeoff here, of course, is performance due to the runtime overhead introduced by decrypting the ticket on each request.

At the very least, you should prevent the ticket from being modified or created on the client by enabling Message Authentication Code (MAC) validation checks. This check results in a hash signature of the ticket contents appended to the ticket, which allows the server to make sure that nobody interfered with the ticket by recomputing the hash and validating it against the signature.

If you turn on encryption for the aforementioned features, you will automatically get the benefits of MAC validation. In addition, ViewState provides the ability to inject a unique value used during validation to make sure that the view state blob was generated for the current user, which prevents other users from being able to submit a form on another user's behalf. This is accomplished by programmatically setting the page ViewStateUserKey property to a user-unique value, such as a session ID or authenticated user name, on every request.

All of these features allow you to downgrade the protection from encryption to validation only, which does not protect the data from being discovered, but does protect it from being changed on the client. This addresses the third threat, a malicious user crafting a custom ticket to impersonate another user.

If you have custom data you would like to store in a way that prevents it from being discovered or modified by the client, you have a few options. You can store it in the session, in which case the state is stored on the server and the client only gets the session ID. You can also store the data in the userData region of the encrypted forms authentication ticket, in ViewState, or in a custom cookie with your own MAC validation and encryption mechanism, although this requires you to manage your own keys.

With both validation and encryption enabled, you still do not address the ticket replay threat, which results from malicious clients being able to steal a valid ticket and later present it to the server, masquerading as the original client to which the ticket was issued. Another variation of this threat is a malicious client forcing the legitimate client to use the ticket provided by the malicious user. There are two general strategies that can help you reduce this risk: preventing the tickets from being stolen, and preventing tickets from being replayed.

To keep tickets from being stolen in the first place, the application should prevent the ticket from being sniffed from the network path to the client, and also ensure that the ticket is not stolen at the client. Protecting the network path is best done by enabling SSL for the entire Web site domain, or at least for the scope of authenticated URLs and the login page. If you do the latter, make sure that the path for the forms authentication, Role Manager, or custom cookie-based ticket is set to the SSL-protected segment of your URL namespace. Setting the path is achieved by either setting the path and cookiePath attributes for forms authentication and Role Manager in their respective configuration sections, or programmatically by setting the Path attribute on the System.Web.HttpCookie object that represents the cookie issued by your application. You can also intercept cookies set by other components by examining the HttpResponse.Cookies collection, and forcefully set the path on those cookies before the response goes out to the client. Note that if you are using URL-based tickets, there is no sure-fire way to ask the client to restrict these tickets to a particular URL namespace. The least you can do there is avoid generating the links to URLs not covered by SSL in your application content.

Keep in mind that SSL introduces a performance penalty, so the coverage decision will depend on the requirements of your application. Using SSL this way ensures that the ticket is not sent in clear text on any portion of the application that can receive the ticket cookie (which is issued for the entire domain by default)or can be linked to with the URL-based, ViewState, or form-stored state. To learn how to enable SSL in IIS, see the Knowledge Base article How To Enable SSL for All Customers Who Interact with Your Web Site in Internet Information Services.

In ASP.NET 2.0, forms authentication and Role Manager features also support the requireSSL and cookieRequireSSL configuration options, respectively, which allow you to mandate SSL usage for all URLs that can generate and accept tickets for these features. If you are using SSL, set these options to true to prevent accidental disclosure of the tickets when they are issued or returned over clear text connections.

Unfortunately, even if you use SSL, the tickets may be stolen on the client. This can happen on public machines, when the browser windows are left open or URLs containing URL-based tickets are bookmarked. It can also happen due to cross-site scripting attacks that can steal cookies (more on these later).

Securing public computers should begin by encouraging your users to close their browser windows when they are finished with the site, and making the Log Out button highly visible and readily available. In addition, you can consider implementing a custom JavaScript solution to detect an idle client session and pop up a window asking whether the user is still there (and if so, making a request to the server to reset the timeout timer). This practice works well with shorter server ticket timeouts. A more extreme solution is available to the forms authentication ticket, allowing you to disable the renewal of the ticket and force the client to log in again after a certain period of time regardless of their activity. This prevents the malicious client who managed to capture the ticket from using it indefinitely by renewing it, because the ticket will always expire in a fixed amount of time. This is done by setting the slidingExpiration attribute in the <system.web><authentication><forms> element to false.

In ASP.NET 2.0, cookie ticket-based features will automatically issue the cookie with the HttpOnly attribute set, which prevents some browsers, including Internet Explorer, from making the cookie available to JavaScript. This effectively prevents cross-site scripting attacks aimed at stealing these cookies from clients using these browsers.

Another best practice is to avoid using URL-based tickets unless you absolutely require them (for example, if you have mobile users or a requirement to support clients that disable cookies). This is because URL-based tickets are more susceptible to being discovered (bookmarked or e-mailed URLs) and to being used in a "phishing" attack (sent by e-mail or posted on a message board) to make another user share a ticket with a malicious client. The best thing to do is use cookie-based tickets for all your features, but if you cannot do that, use one of the auto-detection settings available in ASP.NET 2.0 for all ticket-based features, including forms authentication, Role Manager, and Session State. The two modes available are UseDeviceProfile (the recommended approach), which uses the client browser capabilities to determine whether cookies are supported, and AutoDetect, which performs a redirect-based auto-detection of cookie support before determining the mode. This allows you to scope the URL-based ticket usage only to clients that require it.

Preventing Replay

A few strategies are available to prevent replay of tickets. ASP.NET 2.0 session state provides an added level of protection for URL-based tickets by allowing you to reject and regenerate the session ID ticket when a corresponding session is not found on the server. This prevents multiple clients from sharing the same session ID if they happen to click on the same URL, which may have been indexed by a search engine or posted by a malicious client. This feature is enabled by default and can be set in the regenerateExpiredSessionId attribute of the <sessionstate> configuration element.

You can also attempt to validate that the client providing the ticket is the original client to whom the server issued the ticket. This practice requires you to capture unique attributes of the client and encode them into the ticket (with encryption and hash validation). Then, on each request, you can compare the attributes of the client with the encoded attributes in the ticket, rejecting the ticket if they don't match. Unfortunately, the best you can do with standard HTTP clients is the user agent request header and the client IP address. Both of these are non-unique, especially the IP address if the client is coming from behind a proxy server, used by a number of access providers including AOL, but may provide some additional level of protection. For an example on how to do this, see Jeff Prosise's MSDN Magazine column at Wicked Code: Foiling Session Hijacking Attempts.

Preventing Attacks Through Input Validation

If you take away anything from this article, it should be the age-old security principle that all input is evil! This is especially true with Web applications, most of which receive input from remote anonymous clients. In designing your application, keep in mind that you cannot expect any level of security from the client browser. Just imagine me opening a telnet session and typing in the HTTP request manually. The client can send you any URL, headers, cookies, and form fields completely independent of how you render your pages and your JavaScript.

Because of this, keep in mind that any data your application receives from the Cookies, Headers, QueryString, and Form fields collections may not be safe. You should always validate all such input before making any sensitive decisions based on it. Keep in mind that even if your application does not appear to make any security-sensitive decisions based on the input, the components it uses might. Also, if you are storing this input in a trusted persistence medium, such as a file or database, it may be used to compromise another software system that consumes that data later.

Input validation is best done by defining what correct input to your application looks like, and rejecting all other input that does not meet this definition. This may include type validation, string length validation, type-specific range validation, and regular expression format validation. The stricter your validation, the better.

For input coming from ASP.NET form controls, you can use associated ASP.NET Validator controls to perform a lot of this validation (just make sure to always enforce server-side validation by calling Page.IsValid). For all other data, perform validation in your page callbacks. A good rule of thumb is to perform the validation and reject invalid input as early as possible, but you must still validate input right before it's used.

Some input data can take many forms that mean the same thing, such as Windows user names (user@domain.com or domain\user) or, even more importantly, file paths and URLs. When dealing with such data, it is critical that it is converted to a canonical format before any authorization or resource access decisions are made based on it. This is known as canonicalization, and is frequently the source of many security vulnerabilities in software. Problems occur when two separate software layers, typically the authorization and resource access layers, make different interpretations of a non-canonical string.

To make sure that your application does not fall victim to this problem, it is best not to accept such data from the client, and instead manage it internally in your application. If you do accept such input, you need to convert input to a standard representation, and make sure that you use a standard set of APIs to process it across the different layers of your application. For example, if you are dealing with file system paths, always use the System.IO.Path.GetFullPath APIs to obtain a canonical representation of the path, and use other APIs on that class to work with the resulting path. (Note that native IO APIs and .NET IO APIs use a different canonicalization format, so passing .NET path strings to native code may lead to canonicalization vulnerabilities.) Use the ASP.NET 2.0 System.Web.VirtualPathUtility class to work with ASP.NET virtual paths. And avoid doing your own string manipulation of path data because it is quite possible that you could miss a special case and do it wrong.

The two most common input threats to a Web application are cross-site scripting and SQL injection. Cross-site scripting, also known as XSS, refers to forcing a browser to run client-side code by submitting an encoded script fragment as input to a Web application that echoes it back in a response, not necessarily to the same client. XSS can be used to steal authentication cookies, or to modify the content displayed by the browser with malicious intent. XSS attacks are mitigated by rejecting any input that may contain XSS scripts as part of your input validation strategy, and HTML-encoding the input that gets echoed back to the client to render the XSS scripts completely harmless.

ASP.NET 1.1 provides the validateRequest option that detects common XSS signatures in input to ASP.NET pages and terminates the request. If you can, keep this option enabled on your pages, but keep in mind that it does not apply to non-page content, and this option alone is not sufficient to provide input validation for your application.

If you are writing the response manually or developing a custom control, always HTML-encode the output of your app that is determined to contain client input by using the HttpUtility.HtmlEncode and HttpUtility.UrlEncode methods. ASP.NET 2.0 GridView and DetailsView controls provide the ability to HTML-encode parts of their output that is data bound. However, in all other cases you are responsible for HTML-encoding the strings considered unsafe before setting properties that affect control output.

As a defense-in-depth measure, some ASP.NET 2.0 features set the HttpOnly attribute on its cookies to prevent them from being accessible by client-side script. You should also set this property on your custom cookies.

SQL injection refers to another form of script injection, only this time through injecting specially escaped pieces of SQL queries into input parameters that are used by an application to construct dynamic SQL queries. This can be a very dangerous attack, as it can allow the attacker to steal sensitive data in your database and in some cases even modify the data.

To prevent SQL injection attacks, in addition to using strong input validation, you must not build dynamic SQL queries based on client input. Instead, use fixed queries or stored procedures with strongly typed parameters that are initialized with client input. For an example on how to do this, see Data Security: Stop SQL Injection Attacks Before They Stop You. And for a hard look at how SQL injection attacks can lead to major security vulnerabilities, see Jesper Johansson's TechNet Magazine article at Anatomy of a Hack: How A Criminal Might Infiltrate Your Network.

ASP.NET and CAS

ASP.NET provides a code access security (CAS) model that builds on the code access security mechanisms in the .NET Framework. If you are not familiar with the .NET CAS model, see Code Access Security and the article by Mike Downen in this issue of MSDN Magazine. ASP.NET primarily uses its CAS model to sandbox Web application code, both as a least-privilege mechanism and also to isolate multiple ASP.NET applications on a shared server.

ASP.NET CAS is based on trust levels (see Figure 8), which can be configured for each application by setting the trustLevel attribute in the <system.web><trust> configuration section. The available trust levels are defined in the <system.web><securityPolicy> configuration section, and map each trust level entry to a policy file that contains the corresponding permissions given to the application in that level (the files are located in the .NET Framework version\CONFIG directory). Both sections should be locked in the machine-level configuration when the application is untrusted to prevent the application from being able to override the trust-level setting in its Web.config file. The exact list of permissions that are available in ASP.NET 1.1 trust files can be found in the Trust Levels section of the patterns & practices article at Chapter 9 – Using Code Access Security with ASP.NET.

Figure 8 Default CAS Trust Levels (Close Figure 8)

Trust Level | Description
Full | The application has unrestricted access to all APIs, including calling native code (default).
High | The application cannot call native code, but can do almost everything else.
Medium | The application is sandboxed to prevent it from accessing resources outside of its control, such as accessing files outside of the application root or opening network connections.
Low | The application can use most ASP.NET features, but is restricted from accessing resources outside of its control. This is the trust level suited for untrusted applications on a hosted server.
Minimal | The application is severely restricted. In this level, applications are intended to use built-in ASP.NET features and little or no custom code.

At run time, the set of permissions in the trust policy file is associated with all user code, including compiled assemblies in the /Bin directory, custom page and control code, and code in the App_Code directory. To be precise, the resulting set of permissions is an intersection of the application-level permissions specified in the ASP.NET trust policy file with the permissions in the enterprise, machine, and user levels of the CAS configuration hierarchy, where each lower level can only reduce the set of permissions granted by the level above.

By using trust levels with more restrictive permission sets, you can lower the damage potential of the application if it is compromised, and you can sandbox the application by restricting it to operations that cannot affect other applications on the computer. It is recommended that you test and run your applications in the medium trust level if they are trusted, and low trust level if they are not trusted (especially on a shared server). Be mindful that running in partial trust does introduce a performance penalty so it may not be appropriate in some scenarios. By default, ASP.NET uses the full trust level, which allows the application to have unrestricted access on the machine subject only to the limitations of Windows security.

Your application may encounter several common problems due to insufficient permissions when running in partial-trust. A number of system assemblies require the caller to be fully trusted, by omitting the Allow Partially Trusted Callers Assembly (APTCA) attribute, which effectively makes them not usable in partial trust. In this situation, you have the option of either removing the usage of these assemblies from your application or encapsulating the required functionality of these assemblies in a wrapper assembly that is registered in the Global Assembly Cache (GAC) and has the APTCA attribute set, and asserting the unrestricted permission in the wrapper. If you choose to do the latter, you must be extremely careful to validate and restrict the usage of the wrapped assembly to the minimal required level, and demand another set of permissions to reduce the surface area of the new assembly.

Other code used by your application may demand permissions at run time that are not granted by the active policy file. In this situation, you still have the previous option, but you can also add the minimal level of the needed permission to the desired policy file. This will likely be required for any code that demands custom permissions that the default ASP.NET partial-trust files do not know about. For more information on pursuing either of these options, see Chapter 9 – Using Code Access Security with ASP.NET.

ASP.NET trust levels do not apply to internal ASP.NET code, which always executes with unrestricted permissions. This means that your application can potentially be configured to perform actions that are prohibited for custom code, primarily through configuration settings or declarative settings in ASP.NET pages and controls. For example, even though you cannot connect to SQL Server from your own code, you can configure the SqlDataSource control to connect and execute queries because it is part of the System.Web assembly and is registered in the GAC. ASP.NET 2.0 mitigates this problem by providing an option to execute the entire page processing code path in the application trust, including the internal page processing code path and the associated ASP.NET control processing. This is configured with the processRequestInApplicationTrust attribute in the <system.web><trust> configuration section, and is enabled by default. This setting may cause some of your code to break when it is registered in the GAC due to the assumption that it will execute with unrestricted permissions.

To deal with this issue, and also to simplify security management of permissions, a number of ASP.NET features assert all permissions required for them to execute and instead demand that the app be running in a particular trust-level, as expressed by the AspNetHostingPermission granted in each of the trust-level policy files. For example, the session state feature can be used in all trust levels, but the SQL and State Server out-of-process state storage modes require medium trust and assert all other permissions (such as SqlClientPermission and SocketPermission, respectively).

Note that CAS is completely independent from the Windows security model. CAS uses permissions granted to a piece of code, and Windows uses privileges and object ACLs expressed in terms of the Windows identity of the process or thread. CAS serves to further restrict the set of actions that managed code in a Windows process can perform, but it does not supersede Windows security or allow managed code to bypass it. CAS can only control access to managed code APIs. If an application can call native code, it can bypass all CAS restrictions. Because of this, you should always use a least-privileged Windows identity for your worker processes.

Auditing with Web Events

Auditing is the process of monitoring application usage and collecting security-relevant information. It is an important part of application operations because it allows you to detect and stop potential security breaches, as well as investigate them after the fact so you can prevent them in the future. In some cases, auditing can help you to identify and take action against the attacker.

In ASP.NET 2.0, the application auditing infrastructure is provided through the Web events feature. Web events are a framework for emitting rich informational events during the execution of the application, and channeling that information to one or more configurable providers with rich subscription controls defining how and where the events are sent. ASP.NET provides a base hierarchy of event classes that are used by the framework itself to provide useful information about the execution of an application, and can also be used as a foundation for custom events raised by your application code. One subset of such events that is interesting from a security perspective is the security audit events, which derive from the System.Web.Management.WebAuditEvent class, as described in Figure 9.

Figure 9 Security Audit Events in ASP.NET 2.0 (Close Figure 9)

Event | Description
WebAuditEvent | Base class for all audit events.
WebFailureAuditEvent | Base class for all failure audit events, which are raised whenever a security action fails. ASP.NET 2.0 raises these events when URL or file authorization fails to authorize the request.
WebSuccessAuditEvent | Base class for all success audit events, which are raised whenever a security action succeeds. ASP.NET 2.0 raises these events when URL or file authorization succeeds in authorizing the request.
WebAuthenticationFailureAuditEvent | Base class for authentication failure audit events, which are raised whenever an authentication attempt fails. ASP.NET 2.0 raises these events when Membership providers or forms authentication fail to validate provided credentials, or when the forms authentication ticket fails to be decrypted or validated.
WebAuthenticationSuccessAuditEvent | Base class for authentication success audit events, which are raised whenever an authentication attempt succeeds. ASP.NET 2.0 raises these events when Membership providers or forms authentication succeed in validating provided credentials.
WebViewStateFailureAuditEvent | The class for the ViewState failure audit event, which is raised whenever ASP.NET fails to process the ViewState due to an encryption or validation problem.

If you are performing custom application authorization, accessing sensitive resources, or performing any other security-relevant operation, you should consider instrumenting it by creating and raising a custom event derived either from the WebAuditEvent class or from the combination of the WebAuditFailureEvent and WebAuditSuccessEvent classes.

Figure 10 shows a custom event that is raised when an .aspx page fails to validate an input parameter. You can capture the desired audit events by configuring a Web events rule to direct all events that derive from a particular audit base class to a specific provider, and configuring the appropriate event throttling and buffering settings based on your requirements. Event throttling allows you to constrain the rate of events generated by your application by dropping events when the rate exceeds your configuration, and buffering allows you to accommodate event spikes by buffering events in memory and flushing them out in an intelligent manner to protect the event delivery channel. It is highly recommended that you use a provider that supports buffering, such as the System.Web.Management.SqlWebEventProvider, so that you can use more relaxed throttling settings to avoid losing events while still being able to accommodate a large event flow.

Figure 10 Custom Input Validation Event (Close Figure 10)
<%@ page language="c#" %>
<script runat="server">
public class InputValidationFailureAuditEvent :
    System.Web.Management.WebFailureAuditEvent
{
    public const int InputValidationFailureAuditEventCode = 100100;

    protected String paramName;
    protected String failureReason;

    public InputValidationFailureAuditEvent(Object source,
        String param, String reason) : base(
            String.Format("Parameter validation failed: {0}", param),
            source, InputValidationFailureAuditEventCode)
    {
        paramName = param;
        failureReason = reason;
    }

    public override void FormatCustomEventDetails(
        System.Web.Management.WebEventFormatter formatter)
    {
        formatter.AppendLine(String.Format(
            "Validation failure reason: {0}", failureReason));
    }
}

public const int MaxProductId = 1000;

void Page_Load()
{
    int productId;
    InputValidationFailureAuditEvent e = null;

    if (!Int32.TryParse(Request.QueryString["productId"], out productId)
        || productId < 0 || productId > MaxProductId))
    {
        e = new InputValidationFailureAuditEvent(null, "productId",
            "Product id must be numeric and within the " +
            "legal product id range");
        e.Raise();

        ... // stop page processing
    }
    else
    {
        ... // do page processing
    }
}
</script>

Be aware that normal application operation will generate verbose audit successes that may not be very interesting from a security perspective and may contribute to an overly high event rate that will impact the performance of your application or contribute to a denial of service condition. Because Web events expect that throttling settings will lead to event loss under high event-rate conditions, the sequence number in each event will indicate how many events were actually raised when not all events are captured by the provider.

Figure 11 Auditing Event Raised by Invalid Membership Credentials
Figure 11 Auditing Event Raised by Invalid Membership Credentials

By default, only audit failure events are captured to the event log provider, and appear at most one per minute to prevent event log flooding. Figure 11 shows a WebAuthenticationFailureAuditEvent raised by the Membership provider when the application attempted to authenticate a user with invalid credentials. To learn more about using Web events, take a look at the Beta2 Quickstart tutorial at Web Events.

Wrapping It Up

IIS 6.0 and ASP.NET provide a solid Web application platform engineered with security in mind. But it's ultimately the responsibility of your application to follow security best practices to avoid introducing potential vulnerabilities. That means you have to pay attention to these best practices during design, development, and deployment to make sure the application has as strong foundation for protecting itself. The practices given in this article should help you evaluate the critical aspects of securing your application, and give you some ideas regarding how to best take advantage of the platform security features provided by IIS and ASP.NET.

Because new classes of attacks and new ways to exploit software are constantly being discovered, following the principles of least privilege, reduced surface area, and defense-in-depth throughout your application's lifecycle can help you mitigate unforeseeable attacks in the future. More secure design practices are available from "Introduction to Web Application Security" in the patterns & practices guide, a very good source for Web app security guidance that can be used to learn more about specific ideas discussed here. A resource for more ASP.NET 2.0 security best practices can be found at Security Practices: ASP.NET 2.0 Security Practices at a Glance.

Michael Volodarsky is a technical Program Manager on the Web Platform and Tools Team at Microsoft. He owns the core server infrastructure for ASP.NET and IIS and is now focusing on improving the Web application platform in the next-generation Web server, IIS 7.0.


Who Goes There?: Upgrade Your Site's Authentication with the New ASP.NET 2.0 Membership API

Who Goes There?

Upgrade Your Site's Authentication with the New ASP.NET 2.0 Membership API

Dino Esposito and Andrea Saltarello
Parts of this article are based on a prerelease version of ASP.NET 2.0. Those sections are subject to change.
This article discusses:
  • Common authentication techniques for ASP.NET applications
  • The Membership API and provider model in ASP.NET 2.0
  • Role-based security and authorization
This article uses the following technologies:
ASP.NET, Visual Basic
Code download available at: Membership.exe (148KB)
Contents
ASP.NET Authentication
The Seeds of the Membership API
Customizing Authentication with Membership
Using the Membership API
Writing a Custom Provider
The Adapter Pattern
Conclusion

I
n its first release, ASP.NET introduced forms authentication. This provided a powerful framework for handling user authentication in a secure fashion and with an easy-to-use API. The core of forms authentication hasn't changed in ASP.NET 2.0, so most of the tricks and techniques remain usable. However, a lot of work has been done in ASP.NET 2.0 to expose a new membership system that makes it even easier to create more powerful systems for user authentication and management. When you explore the basics of the provider model (upon which this new membership system is based) and its tight integration with forms authentication, you'll inevitably wonder: how does this affect my existing code and database of users and roles?

At first glance, the totally automatic authentication mechanism of ASP.NET 2.0 appears to work only if you rewrite any existing ASP.NET 1.x validation code you may rely on, including password and role management code. Furthermore, it looks like ASP.NET 2.0 requires you to change your user database to comply with the predefined SQL Server™ or Active Directory® scheme. However, these appearances are deceiving.

The keys to ASP.NET 2.0 authentication are the new Membership API (a generic, public API for authenticating users) and the provider model (the underlying specification that makes it possible). In ASP.NET 2.0, authentication is built atop a provider-based, extensible model. This approach lets you easily change the underlying validation code and user management without having to modify the code that interacts with the membership system. This means you can easily adapt your existing code to connect to the authentication machinery of ASP.NET 2.0.

We'll cover the plumbing of the Membership API and its inherently extensible nature, based on pluggable providers. We'll take an existing ASP.NET 1.x authentication mechanism and port it to ASP.NET 2.0, exposing the legacy authentication mechanism through the new Membership API. The code you must write is minimal, and the level of reusability higher than you may expect.

ASP.NET Authentication

In ASP.NET 1.x, implementing forms authentication is a multi-step procedure. First, tweak the <authentication> section of the app's Web.config file so it will register for forms-based authentication. Then write the login page that will be displayed to requesting users. Finally, you must plan an authentication strategy, set up any necessary storage, and implement any required code. After you place the following in the Web.config file, any page request that lacks a valid authentication ticket will be redirected to login.aspx:

<authentication mode="Forms">
   <forms loginUrl="login.aspx" protection="All" />
</authentication>
<authorization>
   <deny users="?"/>
</authorization>

A special HTTP module, named FormsAuthenticationModule, is in charge of this check. This module hooks up to two pipeline events: AuthenticateRequest and EndRequest. When the request is intercepted and ready for authentication, the module looks for an authentication ticket that contains the user's credentials. The authentication ticket is an encrypted HTTP cookie with a configurable name (.ASPXAUTH by default). If no valid ticket is found, the request is redirected to the specified login page, whose URL has been read from the configuration file. (For information about implementing this process without the cookie, see the sidebar "Cookieless Authentication".)

The login page is a Web Form that contains, at a minimum, two textboxes and a Submit button. There's no predefined template or mandatory layout for the login page; each app can provide the login page that best suits its needs. One essential element, however, is the code bound to the postback event—typically, the Click event of a Button or LinkButton. This gathers user credentials (as well as other available data, such as a selected Remember Me checkbox) and validates them against the database of users.

Figure 1 shows the typical ASP.NET 1.x code. At the heart is the AuthenticateUser custom method. It receives the user name and password and runs a query against the database of users to determine whether the name and password represent a valid account.

Figure 1 ASP.NET 1.x Login Page (Close Figure 1)
Public Class LoginForm
  Inherits System.Web.UI.Page

    Protected errorMsg As System.Web.UI.WebControls.Label
    Protected userName As System.Web.UI.WebControls.TextBox
    Protected passWord As System.Web.UI.WebControls.TextBox
    Protected Button1 As System.Web.UI.WebControls.Button
    Protected Form1 As System.Web.UI.HtmlControls.HtmlForm

    Private Sub Button1_Click( _
      ByVal sender As Object, ByVal e As EventArgs)
        Dim user As String = userName.Text
        Dim password As String = passWord.Text

        ' Custom authentication
        If AuthenticateUser(user, password) Then
            FormsAuthentication.RedirectFromLoginPage(user, false)
        Else
            errorMsg.Text = "Sorry, that's not it."
        End If
    End Sub

    Private Function AuthenticateUser( _
        ByVal username As String, ByVal password As String) As Boolean

        Dim connString As String = GetConnectionString()
        Dim cmdText As String = _
            "SELECT COUNT(*) FROM users WHERE " + _
            "CAST(RTRIM(username) AS VarBinary)=" + +
            "CAST(RTRIM(@theUser) AS VarBinary) " + _
            "AND CAST(RTRIM(password) AS VarBinary)=" + _
            "CAST(RTRIM(@thePassword) AS VarBinary)"
        Dim conn As SqlConnection = New SqlConnection(connString)
        Dim cmd As SqlCommand = New SqlCommand(cmdText, conn)
        cmd.Parameters.Add("@theUser", _
            SqlDbType.NVarChar).Value = username
        cmd.Parameters.Add("@thePassword", _
            SqlDbType.NVarChar).Value = password
        conn.Open()
        Try
            Dim found As Integer = CType(cmd.ExecuteScalar(), Integer)
        Finally
            conn.Close()
        End Try

        Return (found > 0)
    End Function

    Private Function GetConnectionString() As String
        Return "SERVER=localhost;DATABASE=myUserDatabase;" + _
            "Integrated Security=SSPI;"
    End Function

End Class

This sort of code is fairly common and, most importantly, it works well. So what's the problem? If you're engaged in the upgrade of an existing application, you can either ignore the new and timesaving features of ASP.NET 2.0 or drop most of the existing code and rewrite it to address a different database (typically, SQL Server) and schema. But in a reasonably complex application, you will have to deal with much more than just some simple code that verifies credentials. Instead, you'll likely have a user manager object, e-mail and role management, a question/answer protocol for password reminders and resets, password policies, and so forth.

The importance of adhering to the ASP.NET 2.0 programming style is not a matter of functionality. Whether you choose to recompile old-but-still-working code or embrace new-and-hot ASP.NET 2.0 components, the actual behavior of the system, the purpose of which is to authenticate users, will not change. However, by going with the ASP.NET 2.0 model, you will be able to take advantage of new controls and a new, made-to-measure API that spans from membership to role and password management. A bunch of security-related controls (see the sidebar, "Security Controls in ASP.NET 2.0") make building login/logout features a snap. You get ready-made GUI controls that let users log in and out, display login status, change passwords and get reminders, and create new user accounts. These are not just occasional features, but common ones that almost all Web apps require.

Figure 2 ASP.NET 2.0 Configuration Tool
Figure 2 ASP.NET 2.0 Configuration Tool

Most applications also need a back-end system to let administrators create and manage users, passwords, and roles. ASP.NET 2.0 provides new controls for UI-related tasks, an IDE configuration tool (see Figure 2), and a common, extensible API to invoke membership and role management functions.

The Seeds of the Membership API

To understand the real added value of a common, extensible API for common, customizable functions, let's take a deeper look at a security-related feature in ASP.NET 1.x that has recently been the target of criticism: storing passwords in the Web.config file. It's doubtful that any enterprise developers would ever seriously consider storing user information as follows (or, worse yet, in the Web.config file but with passwords as clear text):

<authentication mode="Forms">
   <forms loginUrl="login.aspx" protection="All">
      <credentials passwordFormat="SHA1">
         <user name="Joe" password="07B7F3EE06F278D" />
         <user name="Bob" password="B966BE960E7CBBD" />
      </credentials>
   </forms>
</authentication>
So you may be wondering why ASP.NET would even allow storing user credentials in a configuration file. Well, in doing so, you give the ASP.NET authentication machinery a clear and fixed endpoint where it can query for user data. If you have credentials stored in Web.config, the following code executed within a login page would work quite well:
If FormsAuthentication.Authenticate(user, password) Then
   FormsAuthentication.RedirectFromLoginPage(user, false)
Else
   errorMsg.Text = "Sorry, that's not it."
End If

This code differs from Figure 1 in only one small, yet essential, detail. The task of authenticating users is delegated to a method named Authenticate, which is statically defined in the FormsAuthentication class available from the System.Web assembly.

There are generally two ways to look at this feature. You can think of the Authenticate method as a built-in (and generally not very useful) shortcut to check user names stored in an XML file. Or, a more thoughtful impression is to consider the Authenticate method as the public and immutable interface of a system feature. The availability of an immutable interface makes it possible to create visual components that interact with the page and other controls in a totally automated, codeless, and declarative way.

How would you design a login control from an ASP.NET 1.x perspective? How would you handle the authentication? It seems the simplest trick is to bind the Click event of the Submit button to a user-defined event. The page author handles the event, verifies the passed credentials, and returns true or false. This model works, but requires tight coupling between the page and login form. It also requires some code to be written. Since you're relying on a public and immutable interface to trigger the authentication process, you can embed most of the aforementioned code in the login component. Resulting pages using the login component will be zero-code pages. Of course, it is possible to extend this pattern far beyond the boundaries of the login function.

Customizing Authentication with Membership

A public and fixed API typically means a known and fixed behavior—a strategy and an underlying algorithm. Authentication, for example, means checking credentials. But every application has its own way to do that—through different databases, different schemas, different stored procedures, and so on. As you can see, the same strategy can be implemented through different algorithms.

Figure 3 1.x Scheme
Figure 3 1.x Scheme

In ASP.NET 1.x, the Authenticate method represents a public and fixed API to apply a known strategy, but its implementation is not smart enough to support interchangeable components that customize the way authentication (the actual algorithm) occurs. In ASP.NET 2.0, the Membership API accomplishes a number of tasks that relate to user authentication and management. Unlike the Authenticate method used in ASP.NET 1.x, the Membership API supports the ASP.NET 2.0 provider model, allowing you to specify your own implementation of a given feature. Figure 3 shows the membership scheme for ASP.NET 1.x; Figure 4 shows the same scheme in ASP.NET 2.0. By writing and registering a custom component (the membership provider) you instruct the public Membership API to use your customized methods (as opposed to the default methods) to perform authentication tasks. This model lets you unplug the default implementation of a given feature (user authentication, in this case) and plug in your own.

Figure 4 2.0 Scheme
Figure 4 2.0 Scheme

If you're a software purist, or if you simply like formal terms, these concepts will probably be quite familiar. This is just an instance of the strategy pattern. A strategy pattern indicates an expected behavior (for example, sorting) that can be implemented through a variety of interchangeable algorithms (for example, quick sort or merge sort). Each application then selects the algorithm that fits best. In regard to ASP.NET Membership, the membership provider will implement the expected behavior based on a custom algorithm—Active Directory, a SQL Server database, or an Oracle database, for example.

By registering your membership provider of choice in the Web.config file, you instruct your application to create an instance of an object that it may not actually know at compile time. This model is referred to as the factory pattern.

Using the Membership API

The new Membership class in ASP.NET 2.0 reduces the amount of code you have to write for authenticating users, and it supplies a built-in infrastructure for managing roles. Using the features of the Membership subsystem, you can rewrite the code that authenticates a user, as shown in the following code:

Sub Logon_Click(sender As Object, e As EventArgs)
   Dim user As String = userName.Text
   Dim password As String = passWord.Text
   If (Membership.ValidateUser(user, password))
      FormsAuthentication.RedirectFromLoginPage(user, false)
   Else
      errorMsg.Text = "Sorry, that's not it."
   End If
End Sub
This code is almost identical to that which you would write for an ASP.NET 1.x application, the difference being that this code uses the Membership.ValidateUser static method instead of the FormsAuthentication.Authenticate static method. ValidateUser's actual behavior is determined by the data provider that is registered for use. All providers are required to expose a standard interface, which is rich enough to incorporate several membership-related tasks. ValidateUser provides a wrapper for these APIs, exposing a single method that can be used by your application regardless of which provider is being used. (The provider that should be used is registered in the Web.config file of the application.) You can change the provider, and subsequently change the authentication engine, without having to recompile the application and without having to change any of the code in the page that makes use of the Membership API.

Using the membership capabilities of ASP.NET 2.0 doesn't require a deep understanding of data storage tools and mechanisms (such as SQL Server, stored procedures, and encryption). While still providing myriad configuration options, the Membership API shields you from the various details of how the credentials are stored and verified. The Membership class contains a few static properties and methods that you use in order to obtain a unique identity for each connected user. This user information can also be used with other ASP.NET services, including role-based functions and personalization features.

Figure 5 describes the various properties available for the Membership class. They are all static properties with fairly simple implementations; each property simply accesses the corresponding member on the current provider.Security Controls in ASP.NET 2.0

ASP.NET 2.0 offers several server controls that make it easier to program security-related aspects of a Web application. These are composite controls that provide a rich, customizable user interface and encapsulate a large part of the boilerplate code and markup you would otherwise have to write repeatedly.

Login control Aside from the graphics and page layout, all login pages look very similar. They contain at least two textboxes (for user name and password), a button to validate credentials, perhaps a "Remember Me" checkbox, and possibly links to click if the user has forgotten his password or needs to create a new account. The Login control provides all of this functionality, including the ability to validate the user against the default membership provider. The appearance of the control is fully customizable through templates and style settings. All UI text messages are also customizable through properties of the class.

LoginName control This control captures the name of the currently logged-in user from the User intrinsic object and outputs it using the current style. Internally, the control builds a dynamic instance of a Label control, sets fonts and color accordingly, and displays the text returned by User.Identity.Name. LoginName has a slim programming interface that consists of one property, FormatString, which defines the format of the text to display.

LoginStatus control This control indicates the state of authentication for the current user. The control's UI consists of a link button to log in or log out, depending on the current user logon state. If the user is acting as an anonymous user—that is, he never logged in—the control displays a link button to invite the user to log in. Otherwise, if the user has already successfully passed through the authentication layer, the control displays the Logout button.

LoginView control The LoginView control lets you aggregate the LoginStatus and LoginName controls allowing you to display a custom UI based on the authentication state and role of the user. The control, which is based on templates, simplifies creation of different UIs that are specific to different states (anonymous or authenticated) and roles.

ChangePassword control This provides an out-of-the-box, virtually codeless solution that enables end users to change their password. The control supplies a customizable user interface and built-in behaviors to retrieve an old password and save a new one. The underlying API for password management is the API supplied by the selected membership provider. The ChangePassword control will work in scenarios where a user may or may not be already authenticated. The control detects if a user is authenticated and automatically populates a user name textbox with the name. Even though the user is authenticated, she will still be required to reenter the current password. Once the password has been successfully changed, the control, if properly configured, may send a confirmation e-mail to the user.

PasswordRecovery control This control represents the form that enables a user to recover or reset a lost password and receive it via e-mail. After you drop the PasswordRecovery control onto a Web Form, you may have to make some changes to the membership environment before it will work, but only if you want password recovery to use GetPassword in order to send the password back to the user. PasswordRecovery will automatically change its behavior if it is configured as enablePasswordReset=true and enablePasswordRetrieval=false (the default out of the box). In this case, assuming requiresQuestionAndAnswer is set to true, the control will challenge the user with a question and answer. If the answer matches the one that was previously stored, the control will call ResetPassword. At this point, the control can either display the newly reset password or e-mail the password (based on a configurable e-mail template) to the user. Note that if the stored password is hashed, retrieving it is completely impossible. In this case, the password can only be reset.

CreateUserWizard control This is designed to provide native functionality for creating a new user using the standard Membership API. The control offers a basic behavior that the developer can extend to send a confirmation e-mail to the new user and add steps to the wizard to collect additional information, such as address, phone number, and roles.

Figure 5 Membership Class Properties (Close Figure 5)

Property | Description
ApplicationName | Gets and sets an optional string to identify the application.
EnablePasswordReset | Returns true if the provider supports password reset.
EnablePasswordRetrieval | Returns true if the provider supports password retrieval.
HashAlgorithmType | Returns a string representing the hash algorithm that will be used for passwords, assuming that the feature has been configured to use hashing (introduced post-Beta 2).
MaxInvalidPasswordAttempts | Returns the maximum number of invalid password attempts allowed before the user is locked out.
MinRequiredNonAlphanumericCharacters | Returns the minimum number of punctuation characters required in the password.
MinRequiredPasswordLength | Returns the minimum required length for a password.
PasswordAttemptWindow | Returns the time window for failed password attempts. If the maximum number of invalid password or password answer attempts is exceeded within this time interval (in minutes), the user is locked out.
PasswordStrengthRegularExpression | Returns the regular expression with which the password must comply.
Provider | Returns an instance of the currently configured provider.
Providers | Returns the collection of all registered providers.
RequiresQuestionAndAnswer | Returns true if the provider requires a password question/answer when retrieving or resetting the password.
UserIsOnlineTimeWindow | Specifies the time window, in minutes, for which the user is considered to be online after user activity.

As its name suggests, the Initialize method ensures that the Membership class is properly configured and aligned with the settings in the Web.config file. The Provider property returns a reference to the membership provider currently in use. The Initialize method and the Provider property are then used internally in many of the public static properties. For example, when you access Membership.PasswordAttemptWindow, Initialize is first called to ensure configuration is complete and then the PasswordAttemptWindow property is delegated onto the current provider:

Public Shared ReadOnly Property PasswordAttemptWindow As Integer
  Get
     Membership.Initialize()
     Return Membership.Provider.PasswordAttemptWindow
  End Get
End Property

As of Beta 2, ASP.NET 2.0 ships with a predefined provider that targets Active Directory (System.Web.Security.ActiveDirectoryMembershipProvider), and another for SQL Server databases and MDF files in SQL Server Express (System.Web.Security.SqlMembershipProvider). The providers for Microsoft® Access, which were available in Beta 1, are no longer included with ASP.NET 2.0, but could return as sample source code in the future. You can obtain the list of registered providers through the Providers collection. It is important to note that some properties are provider-specific and may not be supported in certain providers. And, of course, providers can implement more properties than just the ones discussed in this article.

Figure 6 lists the methods of the Membership class, including methods for creating, updating, and deleting users. Some of the methods return or accept a MembershipUser object, which contains information that is stored in the system about the user. If you want to use a custom data store, such as an existing database, you can create your own provider, making your data store accessible to any application using the Membership API. You can also register multiple providers in an application, allowing the application to select the appropriate provider at run time.

Figure 6 Membership Class Shared Methods (Close Figure 6)

Method | Description
CreateUser | Creates a new user and fails if the user already exists.
DeleteUser | Deletes the user corresponding to the specified name.
FindUsersByEmail | Returns a collection of membership users whose e-mail address matches the specified e-mail address.
FindUsersByName | Returns a collection of membership users whose user name matches the specified user name.
GeneratePassword | Generates a random password of the specified length. Returns a
GetAllUsers | collection of all users.
GetNumberOfUsersOnline | Returns the total number of users currently online.
GetUser | Retrieves the MembershipUser object associated with the current or specified user.
GetUserNameByEmail | Obtains the user name that corresponds to the specified e-mail address. This method assumes that the e-mail address is a unique identifier in the user database.
UpdateUser | Takes a MembershipUser object and updates the information stored for the user.
ValidateUser | Authenticates a user using supplied credentials.

You typically use the Web Site Administration Tool (see Figure 2) for offline management of users and passwords. However, to create a new user programmatically, all you need to do is call the CreateUser method. Let's assume you have a login page with two mutually exclusive panels—one for registered users to login and one for new users to register a new account (see Figure 7).

Figure 7 Separate Login Pages
Figure 7 Separate Login Pages

If you click the link that allows new users to register, the page switches panels and displays the input form for collecting the name and password of the new user. The click handler for the Add button runs the following code:

Sub AddNewUser_Click(sender As Object, e As EventArgs)
  Membership.CreateUser(NewUserName.Text, NewUserPassword.Text)
  NewUserPanel.Visible = False
  LogUserPanel.Visible = True
End Sub

Note that the overload of CreateUser used in this example won't actually work with the default membership configuration. While the default configuration enables password-based resets, it requires a question and answer for this operation. As a result, when using the default membership configuration, you'll need to use one of the other overloads.

To delete a user, you use the DeleteUser method, which accepts the registered user name of the user to be deleted:

Membership.DeleteUser(userName)
Retrieving information about a particular user is just as easy, using the GetUser method. This method takes the user name and returns a MembershipUser object:
MembershipUser user = Membership.GetUser("AndreaS")

Once you've got a MembershipUser object, you know all you need to know about a particular user, and you can, for example, programmatically change the password. An application commonly needs to support several operations on passwords, such as changing or resetting it, or sending it to a user (possibly with a question/answer challenge protocol). These functions are all supported by the Membership API, but not necessarily by the underlying provider. If the provider does not support a given feature, an exception is thrown when the method is invoked.

To use the ChangePassword method, you must supply the old password in addition to the new password:

   Dim user As MembershipUser = Membership.GetUser("AndreaS")
   user.ChangePassword(user.GetPassword(), newPassword)
This, however, might not work in some implementations, such as if you configured the membership system to store the passwords in your database hashed and/or salted—a common security measure. In this case, the original password cannot be retrieved, and resetting the password is the only option available if the user has forgotten the current password. To reset the password, you use the ResetPassword method:
Dim user As MembershipUser = Membership.GetUser("AndreaS")
string newPassword = user.ResetPassword()

The subsystem of your application that calls ResetPassword is also in charge of sending the new password to the user (for example, via e-mail). Both the GetPassword and ResetPassword methods have a second overload that takes a string parameter. If specified, this string represents the answer to the user's "forgot password" question.

The ability to reset a password, as well as support for a password challenge, is specific to the provider and is configured in the Web.config file. The password challenge question is exposed as a read/write member of the MembershipUser class. When the user is created, the challenge question and answer must be set and stored in the membership database.

Writing a Custom Provider

If you're fine with storing membership data in a SQL Server Express database file and with the standard schema adopted by ASP.NET, there's not much else to do other than using the Web Site Administration Tool to create and fill the database for the current application. But what if you want to use an Oracle database or a SQL Server database with a custom schema? Or, more importantly, what if you invested time and money in building a full membership system for your current ASP.NET 1.x application? Not to worry. You can reuse all or part of this code—just as long as you are willing and able to write a custom membership provider.

All the providers used in ASP.NET 2.0 implement a common set of members; those defined by the ProviderBase abstract class. The class comes with one virtual method (Initialize) and two virtual properties (Name and Description). The Name property returns the official name of the provider class. The Initialize method takes the name of the provider and a name/value collection object packed with the content of the provider's configuration section. A number of abstract classes derive from ProviderBase and act as the base classes for a variety of specific provider tasks, including MembershipProvider, RoleProvider, and a few others.

MembershipProvider is not directly usable, because it consists of several abstract methods. To build a custom membership provider, you inherit from MembershipProvider. Figure 8 provides the list of virtual properties and methods to override.

Figure 8 MembershipProvider Class (Close Figure 8)
Public MustInherit class MembershipProvider
  Inherits ProviderBase

    ' Events
    Public Event ValidatingPassword As _
        MembershipValidatePasswordEventHandler

    ' Methods
    Public MustOverride Function ChangePassword( _
        ByVal username As String, ByVal oldPassword As String, _
        ByVal newPassword As String) As Boolean
    Public MustOverride Function ChangePasswordQuestionAndAnswer( _
        ByVal username As String, ByVal password As String, _
        ByVal newPasswordQuestion As String, _
        ByVal newPasswordAnswer As String) As Boolean
    Public MustOverride Function CreateUser( _
        ByVal username As String, ByVal password As String, _
        ByVal email As String, ByVal passwordQuestion As String, _
        ByVal passwordAnswer As String, ByVal isApproved As Boolean, _
        ByVal providerUserKey As Object, _
        <Out> ByRef status As MembershipCreateStatus) As MembershipUser
    Public MustOverride Function DeleteUser( _
        ByVal username As String, _
        ByVal deleteAllRelatedData As Boolean) As Boolean
    Public MustOverride Function FindUsersByEmail( _
        ByVal emailToMatch As String, ByVal pageIndex As Integer, _
        ByVal pageSize As Integer, <Out> ByRef totalRecords As Integer) _
        As MembershipUserCollection
    Public MustOverride Function FindUsersByName( _
        ByVal usernameToMatch As String, ByVal pageIndex As Integer, _
        ByVal pageSize As Integer, <Out> ByRef totalRecords As Integer) _
        As MembershipUserCollection
    Public MustOverride Function GetAllUsers( _
        ByVal pageIndex As Integer, ByVal pageSize As Integer, _
        <Out> ByRef totalRecords As Integer) As MembershipUserCollection
    Public MustOverride Function GetNumberOfUsersOnline() As Integer
    Public MustOverride Function GetPassword( _
        ByVal username As String, ByVal answer As String) As String
    Public MustOverride Function GetUser( _
        ByVal providerUserKey As Object, ByVal userIsOnline As Boolean) _
        As MembershipUser
    Public MustOverride Function GetUser( _
        ByVal username As String, ByVal userIsOnline As Boolean) _
        As MembershipUser
    Public MustOverride Function GetUserNameByEmail( _
        ByVal email As String) As String
    Public MustOverride Function ResetPassword( _
        ByVal username As String, ByVal answer As String) As String
    Public MustOverride Function UnlockUser( _
        ByVal userName As String) As Boolean
    Public MustOverride Sub UpdateUser(ByVal user As MembershipUser)
    Public MustOverride Function ValidateUser( _
        ByVal username As String, ByVal password As String) As Boolean

    ' Properties
    Public MustOverride Property ApplicationName As String
    Public MustOverride ReadOnly Property EnablePasswordReset As Boolean
    Public MustOverride ReadOnly Property EnablePasswordRetrieval _
        As Boolean
    Public MustOverride ReadOnly Property MaxInvalidPasswordAttempts _
        As Integer
    Public MustOverride ReadOnly Property _
        MinRequiredNonAlphanumericCharacters As Integer
    Public MustOverride ReadOnly Property _
        MinRequiredPasswordLength As Integer
    Public MustOverride ReadOnly Property PasswordAttemptWindow As Integer
    Public MustOverride ReadOnly Property _
        PasswordFormat As MembershipPasswordFormat
    Public MustOverride ReadOnly Property _
        PasswordStrengthRegularExpression As String
    Public MustOverride ReadOnly Property _
        RequiresQuestionAndAnswer As Boolean
    Public MustOverride ReadOnly Property RequiresUniqueEmail As Boolean
End Class

Figure 9 shows an extremely simple, but usable sample class that incorporates the authentication code used in Figure 1 for an ASP.NET 1.x application. As you can see, most methods and properties have a null implementation that just raises an exception. The implementation for ValidateUser, however, calls back into some code imported from the ASP.NET 1.x application. The more features you have, the more you can port to ASP.NET 2.0 by overriding other methods and properties in a more meaningful way. For example, if your existing code includes a user manager object to perform Create/Read/Update/Delete (CRUD) operations on users, you can easily map those members to the methods of the custom membership provider. The companion code for this article, which can be downloaded from the MSDN®Magazine Web site, demonstrates how to port such functionality to ASP.NET 2.0.

Figure 9 Simple Custom Membership Provider (Close Figure 9)
Imports System
Imports System.Data
Imports System.Data.SqlClient
Imports System.Web
Imports System.Web.UI
Imports System.Web.Security

Namespace MsdnMag
    Public Class MyMembershipProvider : Inherits MembershipProvider

        Public Sub New()
        End Sub

        Private _applicationName As String = "SampleApp"

        ... ' Members ommitted that simply throw a NotsupportedException

        Public Overrides Function ValidateUser(ByVal username As String, _
                ByVal password As String) As Boolean
            Return AuthenticateUser(username, password)
        End Function

        Public Overrides Property ApplicationName() As String
            Get
                Return _applicationName
            End Get
            Set(ByVal value As String)
                _applicationName = value
            End Set
        End Property

        Public Overrides ReadOnly Property EnablePasswordReset() _
                As Boolean
            Get
                Return False
            End Get
        End Property

        Public Overrides ReadOnly Property EnablePasswordRetrieval() _
                As Boolean
            Get
                Return False
            End Get
        End Property

        Public Overrides ReadOnly Property _
                MinRequiredNonAlphanumericCharacters() As Integer
            Get
                Return 1
            End Get
        End Property

        Public Overrides ReadOnly Property MinRequiredPasswordLength() _
                As Integer
            Get
                Return 12
            End Get
        End Property

        Private Function AuthenticateUser( _
             ByVal username As String, ByVal password As String) As Boolean
            ... ' code from AuthenticateUser in Figure 1
        End Function
    End Class
End Namespace

To register the new membership provider, you need to tweak the Web.config file as shown here:

<membership defaultProvider="MyMembershipProvider">
   <providers>
     <add name="MyMembershipProvider"
          type="MsdnMag.MyMembershipProvider" />
   </providers>
</membership>
Simply add this code to your application's Web.config file under the <system.web> node and you're done. From now on, any call to the Membership class will be resolved calling into the methods of your provider class. Figure 10 shows the Web Site Administration Tool being used to select the new provider.

Figure 10 Managing the New Membership Provider
Figure 10 Managing the New Membership Provider

The Adapter Pattern

You've now seen that ASP.NET 1.x membership code can be properly retooled and reused in ASP.NET 2.0 applications. No matter how complex your existing authentication engine might be, there is always a way to reuse large portions and integrate these pieces with the cool, new ASP.NET 2.0 security and authentication infrastructure. Essential to achieving these results is another popular design pattern: the adapter pattern.

The adapter pattern converts the programming interface of some class A into an interface B that some client class C fully understands. In other words, the adapter pattern designates a class that wraps the implementation of another class and exposes this second class via a different interface. In this context, the most exciting characteristic of the adapter pattern is that it provides a way for the membership provider class to permit access to its internal implementation (based on existing ASP.NET 1.x code) in such a way that a caller (such as ASP.NET 2.0 login pages or security controls) is not bound to the specifications of that implementation.

On a side note, the adapter pattern can also be used to reuse some functionality out of a class without resorting to inheritance. To paraphrase Erich Gamma and his co-authors in Design Patterns (Addison-Wesley, 1995), you should always favor object composition over inheritance. Inheritance should be primarily used as a way to specialize existing types to take advantage of polymorphism as opposed to a strategy for reusing existing code. Moreover, in a single-inheritance environment like the .NET Framework, this pattern gains importance as it allows you to save the base class for other situations.

Applying the adapter pattern will be easier if you better organize your ASP.NET 1.x code today. Wrapping all of your logic and data into a bunch of classes can only help. At a minimum, you can have a User class that represents the user of your system with all the expected properties and any assigned behaviors. In addition, you can have a collection of User objects and a manager class to perform CRUD tasks. Depending on the desired level of sophistication, you can also have Password and Role objects with their own APIs. Rewriting ASP.NET 1.x code as classes is not mandatory, but it will make it significantly easier to port the code to ASP.NET 2.0.

Conclusion

The ASP.NET 2.0 Membership API provides an extensible model for building your own validation layer to handle user credentials. This is a great benefit because you're not limited to using a particular data store and schema. In addition, there is an array of new server controls available to simplify the creation of security-related user interfaces (see the sidebar on "Security Controls in ASP.NET 2.0" for more information).Cookieless Authentication

In ASP.NET 1.x, cookies are mandatory if you want to take advantage of the built-in authentication framework (assuming you aren't using Microsoft Mobile Internet Toolkit in ASP.NET 1.1). In ASP.NET 2.0, the core forms authentication API also supports cookieless semantics. More precisely, the entire API has been reworked so that it exposes a nearly identical programming interface, but with support for dual semantics: cookied and cookieless.

In theory, there are several ways you can implement cookieless authentication. For instance, you can append the ticket to the query string or store it as extra path information and retrieve it from the PATH_INFO server variable. But ASP.NET 2.0 takes a different approach, which is consistent with the implementation of cookieless sessions. The authentication ticket is packed into the URL, as shown here:

   /SampleApp/(XYZ1234)/default.aspx

ASP.NET intercepts a request, extracts the ticket, and rewrites the correct path to the application. The authentication ticket is then exposed as another request header. The <forms> section of the Web.config file is extended with the cookieless attribute. There are four possible values: AutoDetect, UseCookies, UseDeviceProfile, and UseUri. AutoDetect indicates that cookies are used if the browser has cookie support enabled. Otherwise, it uses the cookieless mechanism. UseCookies specifies that cookies are used regardless of the browser's capabilities. When UseDeviceProfile is set, ASP.NET uses cookies if the browser supports them (as defined in a device profile stored on disk). Otherwise, it uses the cookieless mechanism. With this option, no attempt is made to check whether cookie support is really enabled on the requesting device. Finally, UseUri forces the use of cookieless authentication.

The provider model of ASP.NET 2.0 (an implementation of the strategy pattern) offers an excellent way to avoid having to re-architect your existing solutions. Applied to membership, the provider model allows ASP.NET pages and security controls to connect to custom components that supply predefined interfaces. By applying the adapter pattern, you can take an existing class, typically the entry point to your ASP.NET 1.x membership system, and adapt it to expose any internals via the ASP.NET 2.0 standard interface. In this way, you can plug your old code and data store into the dazzling new world of ASP.NET 2.0. Best of all, you'll have the new features (such as security controls, standard APIs, and integration with the Web Site Administration Tool) but still be able to reuse most of your old code.

Dino Esposito is a mentor at Solid Quality Learning and the author of Programming Microsoft ASP.NET 2.0 (Microsoft Press, 2005) Based in Italy, Dino is a frequent speaker at industry events worldwide. Get in touch with Dino at cutting@microsoft.com or join the blog at weblogs.asp.net/despos.

Andrea Saltarello is the technical director and chief software architect at Managed Designs. He is an ASP.NET MVP and leader of the Italian .NET User Group. Read his blog at blogs.ugidotnet.org.


What Gives You the Right?: Combine the Powers of AzMan and WSE 3.0 to Protect Your Web Services

What Gives You the Right?

Combine the Powers of AzMan and WSE 3.0 to Protect Your Web Services

Niels Flensted-Jensen
This article is based on a prerelease version of Web Service Enhancements 3.0. All information is subject to change.
This article discusses:
  • Securing Web services using Web Service Enhancements 3.0
  • Writing custom policy assertions
  • Integrating with Enterprise Library and Authorization Manager for task-based authorization
This article uses the following technologies:
.NET Framework 2.0, WSE, Authorization Manager, and Enterprise Library
Code download available at: AzManandWSE30.exe (149KB)
Contents
Extensions to Basic Messaging
Trusted Subsystems and Authorization
WSE to the Rescue
Simple Client-to-Service Scenario
MethodAuthorization Policy Assertion
Getting the Assertion into the Policy Pipeline
Filtering Messages Passing Through the Pipeline
Enterprise Library AzMan Provider
Dynamic Authorization
Making Authorization Claims and Enabling the Client
The Bigger Picture

I
n the world of connected Web services, security is a central issue. However, transaction handling, reliable messaging, and performance are other issues that present significant challenges. All of these issues are helped by addressing the need for fine-grained, dynamic authorization. In this article, I show you how to engineer an approach for the declarative specification of authorization requirements for individual service endpoints. I deliberately use the verb "engineer" as opposed to craft or program because my focus here is on building a solid solution from ready-made parts of the platform and tools available.

I demonstrate how you can combine existing and new Microsoft technologies with minimal new code to provide flexible authorization for individual Web service methods. Windows® 2003 Authorization Manager, Web Service Enhancements (WSE) 3.0, and Enterprise Library all play a part. By inserting a simple policy assertion into your policy file, you get task-based authorization that goes beyond what can be achieved by, for example, applying System.Security.Permissions.PrincipalPermissionAttribute to your code. But before producing the solution, let's take a look at the foundation that allows us to do things like this in Web services.

Extensions to Basic Messaging

Central to solving the challenges faced by the service designer and implementer are the many Web service (WS) specifications. Their level of stability and adoption varies greatly. Note, however, when arguing the case for adoption, that only in a few cases are these specifications actual ratified standards. Each specification, such as WS-Addressing or WS-Security, addresses separate concerns in the communication that occurs with and between your services, and as such represents stackable protocols that may be combined as necessary according to your needs. For instance, transport encryption and traditional URLs must be augmented by message-level security (WS-Security, WS-SecureConversation) and addressing (WS-Addressing) when introducing intermediaries into the service infrastructure.

The visible manifestation of the WS specifications will be primarily the introduction of headers into the SOAP envelope carrying the Web service message. But depending on the type of WS protocol, the actual contents of the SOAP body may also be altered, as is the case when applying message-level encryption. Figure 1 shows two examples of the same SOAP envelope. The second version has the WS headers applied along with their corresponding namespaces. Note that the body stays the same. When developing solutions, you will notice the growing complexity of the header section as you start applying various aspects, such as WS-Addressing and WS-Security.

Figure 1 Applying WS Headers to a SOAP Envelope (Close Figure 1)

Without WS Headers
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <soap:Body>
    <ViewInvoices xmlns="http://example.org/invoices" />
  </soap:Body>
</soap:Envelope>
With WS Headers
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema"
    xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/08/addressing"
    xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-
      wssecurity-secext-1.0.xsd"
    xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-
      wssecurity-utility-1.0.xsd">
  <soap:Header>
    <wsa:Action>http://example.org/invoices/ViewInvoices</wsa:Action>
    <wsa:MessageID>
      uuid:3db181be-3053-4f1d-8e2b-f5c22e9a4c44</wsa:MessageID>
    <wsa:ReplyTo>
      <wsa:Address>
        http://schemas.xmlsoap.org/ws/2004/08/addressing/role/anonymous
      </wsa:Address>
    </wsa:ReplyTo>
    <wsa:To>
      http://localhost/SecureInvoiceService/Part1/Before/
      SecureInvoiceService.asmx
    </wsa:To>
    <wsse:Security>
      <wsu:Timestamp
          wsu:Id="Timestamp-c18214c0-8318-4d33-ba21-8820277ac1e4">
        <wsu:Created>2005-07-08T06:34:54Z</wsu:Created>
        <wsu:Expires>2005-07-08T06:39:54Z</wsu:Expires>
      </wsu:Timestamp>
    </wsse:Security>
  </soap:Header>
  <soap:Body>
     <ViewInvoices xmlns="http://example.org/invoices" />
  </soap:Body>
</soap:Envelope>

Trusted Subsystems and Authorization

Before moving on to a discussion of authorization and some example code, let's look at how services are deployed and how they interoperate. Each service should run under the identity of a domain service account created specifically for that service during installation. This way, the clear text password will never be known to any user or machine, and the service may be locked down very tightly with regard to access to file system, registry, network, Active Directory®, and so on. Services may be deployed in a trusted subsystem model in which each service trusts the previous service to perform appropriate authentication. A more granular approach requires performing some custom authorization in the service code. For this, you need to extend the service infrastructure not only to check authorization of the service sending you a message, but also to check the identity of the user or machine that is requesting for the action to be performed.

Figure 2 illustrates the trusted subsystem model inside an Active Directory domain with NTFS security supplementing WS-Security-based authentication and authorization. Note that no matter which user succeeds in accessing the service, the server will never impersonate the user. This allows for tight and predictable lockdown of the service.

Figure 2 Locking Down Services While Passing User Identity
Figure 2 Locking Down Services While Passing User Identity

If you develop ASP.NET Web applications, you know that users may be checked for role-based authorization in various ways and at various levels (application, directory, or page). But this only works for standard .NET Framework Web services if you are using Windows-integrated security. The same restriction applies to IIS 6.0 URL authorization, which takes place even before ASP.NET has a chance to kick in with something like forms-based authentication. And even if you work within the integrated security constraints, you are still not inherently able to transfer, much less authorize, identities on whose behalf a service performs work.

In ASP.NET, the default support for Web services only supports basic scenarios and does not provide the mechanisms and hooks required to build secure service-oriented architectures out of the box. WSE 3.0 addresses this issue, and Windows Server™ 2003 Authorization Manager (AzMan.msc) provides a good platform for doing fine-grained and flexible authorization.

WSE to the Rescue

WSE is a developer-focused security product for Web services. It allows you to implement message-level security without having to focus on the specification details. As I'm writing this, WSE 3.0 is in its Beta release. It promises wire-level compatibility with the Windows Communication Foundation (formerly codenamed "Indigo"). For more details on WSE 3.0 features and for related downloads, see Web Services Enhancements.

WSE features, including the various security functions, are built around the concept of a pipeline. The pipeline is where incoming and outgoing messages are intercepted and validated, transformed, logged, and possibly checked for authorization.

As Figure 3 illustrates, a pipeline handles both input and output messages and is configured entirely by its policy. Policies are made up by one or more policy assertions that apply at both input and output. Each is implemented using a fairly simple PolicyAssertion assertion that contains up to four SoapFilter filters (output client, input client, input service, output service). Here I'll concentrate on the methodAuthorization assertion illustrated in Figure 3.

Figure 3 WSE 3.0 Pipeline and Policy
Figure 3 WSE 3.0 Pipeline and Policy

Note that WSE 3.0 already contains a standard authorization assertion, which provides functionality borrowed from ASP.NET role authorization. But this does not cover individual Web methods or task-based authorization. This feature can be enabled using the WSE 3.0 policy wizard available from Visual Studio® 2005.

Simple Client-to-Service Scenario

To demonstrate the concepts discussed here, my starting point will be the security hands-on lab (WSE 3.0 Hands On Lab—Exploring Security) available from the WSE Web site at Web Services Enhancements. Specifically, I used the Part3_After solution as the starting point for developing and configuring the new authorization features.

My example Invoice Manager uses a simple Windows Forms client application (shown in Figure 4), which communicates with a Web service with four end points: View, Submit, Approve, and Pay. Unlike the sample in the WSE hands-on lab, this version uses Kerberos tokens for both signatures and encryption, thereby removing the need to install a service certificate. My example is put together so that both the client and server simply run on the same machine, but this could easily be modified to do otherwise.

Figure 4 Invoice Manager Client
Figure 4 Invoice Manager Client

The intention is to guard access at the service level to ensure that users attempting to View, Submit, and so on are actually authorized to do so. To this end AzMan is used to provide flexible authorization. AzMan allows for very flexible configuration of access rights in a way fully separated from both client and service development. Figure 5 shows the AzMan interface used to configure authorization for the sample application. Note the Role Assignments node, which is where you will be adding yourself or other users who will test the authorization functionality of the developed WSE 3.0 policy assertion.

Figure 5 Authorization Manager
Figure 5 Authorization Manager

All the different elements of the solution will fit together at run time as illustrated in Figure 6. The client sends messages through its WSE proxy to the service, where the WSE pipeline intercepts the messages to perform further verification, authentication, and authorization. The authorization step will rely on the Enterprise Library AzMan authorization provider, which will access Authorization Manager through the managed AzMan API. This will be based on claims made as part of the configured policy assertion in the policy file. (See the "Enterprise Library" sidebar for background information.)

Figure 6 Sample Authorization Implementation
Figure 6 Sample Authorization Implementation

MethodAuthorization Policy Assertion

Coding your own authorization assertion to insert into the WSE pipeline is straightforward, as is configuring it for use with an actual service policy. The notable risk is configuring the assertion in the wrong pipeline position. Placing it before the authentication assertion will cause the authorization assertion to fail because the identity for authorization has not yet been discovered.

Figure 7 shows the code required to prepare your assertion to be properly hooked into the WSE pipeline. The assertion and its corresponding filters are created once on startup, and the same instances are then used for the duration of the application's lifetime.

Figure 7 MethodAuthorization WSE 3.0 Policy Assertion (Close Figure 7)
public class MethodAuthorizationAssertion : PolicyAssertion
{
    private string authorizationProviderName = null;
    private Dictionary<string, AuthorizationData> actionAzList = null;

    public override SoapFilter CreateClientInputFilter(
        FilterCreationContext context)
    {
        // not applicable to the client pipeline
        throw new NotSupportedException(
            "The MethodAuthorizationAssertion assertion " +
            "is not to be applied on the client." );
    }

    public override SoapFilter CreateClientOutputFilter(
        FilterCreationContext context)
    {
        // not applicable to the client pipeline
        throw new NotSupportedException(
            "The MethodAuthorizationAssertion assertion " +
            "is not to be applied on the client." );
    }

    public override SoapFilter CreateServiceInputFilter(
        FilterCreationContext context)
    {
        return new MethodAuthorizationInputFilter(this);
    }

    public override SoapFilter CreateServiceOutputFilter(
        FilterCreationContext context)
    {
        // nothing to do on the way out
        return null;
    }

    public override void ReadXml(XmlReader reader,
        IDictionary<string, Type> extensions)
    {
        if (reader == null) throw new ArgumentNullException("reader");

        // locate the name of the start element
        string startElementName = null;
        foreach (KeyValuePair<string, Type> pair in extensions)
        {
            if (pair.Value == typeof(MethodAuthorizationAssertion))
            {
                startElementName = pair.Key;
                break;
            }
        }

        bool isEmpty = reader.IsEmptyElement;
        authorizationProviderName = reader.GetAttribute(
            "authorizationProvider");

        reader.ReadStartElement(startElementName);
        if (!isEmpty)
        {
            string actionName = null;
            if (reader.MoveToContent() == XmlNodeType.Element)
            {
                do
                {
                    if (reader.Name != "action")
                        throw new FormatException(
                            "The <methodAuthorization> assertion " +
                            "may only contain <action> elements.");

                    actionName = reader.GetAttribute("name");
                    if (actionName == null || actionName.Length == 0)
                        throw new FormatException("The <action> " +
                            "element must contain a 'name' attribute." );

                    AuthorizationData azData = new AuthorizationData();
                    azData.Rules = reader.GetAttribute("rules");
                    azData.Tasks = reader.GetAttribute("tasks");
                    azData.Operations = reader.GetAttribute("operations");

                    if (actionAzList == null) actionAzList =
                        new Dictionary<string,AuthorizationData>();
                    actionAzList[actionName] = azData;
                }
                while (reader.Read() && reader.MoveToContent() ==
                    XmlNodeType.Element);
            }
            reader.ReadEndElement();
        }
    }

    ... // private SoapFilter defined here
}

To implement a class derived from the abstract PolicyAssertion class, you must implement the four missing filter creation methods. Since I'm creating an assertion to be used only at the service interface for authorization of incoming messages, I'm really only interested in the CreateServiceInputFilter method. Therefore, I return null from CreateServiceOutputFilter.

To configure the authorization assertion with the desired authorization claims, using the Enterprise Library Authorization Provider of my choice, I must override the ReadXml method that reads in the assertion configuration from the policy file.

I will get back to the actual implementation of the SoapFilters shortly, but let's first see how the newly implemented service is configured to fit into your policy pipeline.

Getting the Assertion into the Policy Pipeline

Once you have implemented your new policy assertion, the next step is to configure a policy to include the assertion in the pipeline. First, set up a WSE 3.0 policy for your Web service application to use. Once you have installed WSE 3.0, bring up the Web service project in Visual Studio® 2005 and use the context menu on the project in solution explorer to bring up the WSE configuration tool. Enable both WSE and the WSE SOAP protocol factory. On the Policy tab, add or remove policy definitions as necessary. Note that once a policy is set up, further modifications will have to be done by hand. This is when you will probably appreciate the simplified WSE 3.0 policy configuration format shown in Figure 3. (If you've ever looked at a WS-SecurityPolicy configuration as used in WSE 2.0, you've stared complexity right in the face—and yet you've still managed to stay at it or you wouldn't be reading this!)

First, you tell WSE that you have produced an extension, you give it a name to be referenced by, and then you tell it which type implements the assertion. That said, let me just briefly comment again on the policy format. WS-Policy has been superseded inWSE 3.0 with a simpler format that addresses common security scenarios when developing Web services today.Enterprise Library

Enterprise Library is a framework, developed by the Microsoft patterns & practices team, that consists of a set of application blocks that provide generic solutions to many of the problems you run into every day, such as configuration, security, data access, exception handling, and logging. The Configuration Console saves you from having to edit configuration files manually. As full source code is provided along with the compiled blocks, you're free to modify or extend the application blocks to meet your needs. For more information visit Enterprise Library.

Information about using the sample application with the .NET Framework 2.0 and the June 2005 release of Enterprise Library is provided in the readme file that accompanies the code download for this article.

Filtering Messages Passing Through the Pipeline

After setting up the assertion, I then implement classes derived from Microsoft.Web.Services3.SoapFilter to do the actual filtering (see Figure 8).

Figure 8 Checking the Sender's Authorization (Close Figure 8)
private class MethodAuthorizationInputFilter : SoapFilter
{
    private IAuthorizationProvider azProvider = null;
    private MethodAuthorizationAssertion assertion = null;

    public MethodAuthorizationInputFilter(
            MethodAuthorizationAssertion assertion ) : base()
    {
        if (assertion.authorizationProviderName == null ||
            assertion.authorizationProviderName.Length == 0)
        {
            azProvider = AuthorizationFactory.GetAuthorizationProvider();
        }
        else
        {
            azProvider = AuthorizationFactory.GetAuthorizationProvider(
                assertion.authorizationProviderName );
        }
        if (azProvider == null)
            throw new SecurityFault(
                "Authorization configuration error" );

        this.assertion = assertion;
    }

    public override SoapFilterResult ProcessMessage(
        SoapEnvelope envelope )
    {
        WindowsPrincipal principal =
            envelope.Context.IdentityToken.Principal
                as WindowsPrincipal;
        if (principal == null)
            throw new SecurityFault(
                SecurityFault.SecurityTokenUnavailableMessage );

        if (!CheckAccessToMethod(
            envelope.Context.Addressing.Action.Value, principal ))
            throw new SecurityFault(
                "Not authorized to access this endpoint.",
                SecurityFault.FailedCheckCode );

         return SoapFilterResult.Continue;
    }

    ...
}

MethodAuthorizationAssertion constructs MethodAuthorizationInputFilter to process an incoming SOAP message and check the authorization of the sender.

The constructor of the filter is supplied with a reference to the relevant assertion, which will be used to access the configuration data (the AzMan provider and authorization claims) during message processing. The actual processing takes place in the ProcessMessage method, which will start by picking out the principal representing the identity of the sender of the message. You should note that it will look for a WindowsPrincipal. The AzMan provider that is delivered with Enterprise Library supports only that type, though AzMan may accept any Security Identification (SID) as identification of a principal.

Once the identity of the sender has been established, the rest is simply a matter of checking the authorization claim associated with the WS-Addressing Action header as contained in the incoming SoapMessage (this example makes sure to require an Action header through the <requireActionHeader> policy assertion). Remember from Figure 3 that this claim was configured as part of the policy assertion in the policy file.

Enterprise Library AzMan Provider

When the action header has been retrieved and the identity of the sender established, it's time to perform the actual authorization check. Enterprise Library includes a nice configuration tool and provider factory engine, so I simply applied the Enterprise Library AzMan Provider to this task. The provider implements Microsoft.Practices.EnterpriseLibrary.Security.IAuthorization which has only one method to be concerned with: Authorize. Since the provider has already been hooked up in the constructor of MethodAuthorizationInputFilter, all you need to do is simply issue the call like this:

authorized = azProvider.Authorize( principal, task );

The authorization code implemented by the CheckAccessToMethod method, shown in Figure 9, has several steps that look for Rules, Tasks, and Operations. Rules are specific to another Enterprise Library authorization provider, the Authorization Rule Provider, whereas Tasks and Operations are specific to the AzMan provider. Performance hasn't been a big concern, but if it were, reuse of providers and internal optimization of the AzMan provider might be worthwhile.

Figure 9 Looking for Rules, Tasks, and Operations (Close Figure 9)
private bool CheckAccessToMethod(string action,
    WindowsPrincipal principal ) {
    // No authorization claims has been made
    if (assertion.actionAzList == null) return true;

    bool authorized = true;

    AuthorizationData azData;
    if (assertion.actionAzList.TryGetValue( action, out azData ) &&
        azData != null)
    {
        // Run through Tasks, Operations, and Rules. As long as one
        // evaluates to true the token.Principal is authorized.
        authorized = azData.Rules == null && azData.Tasks == null &&
        azData.Operations == null;
        if (!authorized)
        {
            foreach (string rule in azData.GetRuleList())
                if (authorized = azProvider.Authorize(principal, rule))
                    break;
            if (!authorized)
            {
                foreach (string task in azData.GetTaskList())
                    if (authorized =
                        azProvider.Authorize(principal, task)) break;
                if (!authorized)
                {
                     foreach (string op in azData.GetOperationList())
                         if (authorized = azProvider.Authorize(
                             principal, string.Format("O:{0}", op)))
                                 break;
                }
            }
        }
    }

    return authorized;
 }

It should also be noted that although the AzMan provider supports only WindowsIdentities, it may well be extended to support other SIDs. This will require extending the functionality in the GetClientContext method of the AzMan provider a bit to initialize the AzMan context using IAzApplication.InitializeClientContextFromStringSid.

Dynamic Authorization

So far I have been concerned with how to check whether a principal is authorized for a specific task. But I haven't stopped to describe which principals will actually be authorized. Of course, authorizations are managed by the Authorization Manager Microsoft Management Console (MMC) snap-in, illustrated in Figure 5.

Before getting anything useful from AzMan, the application must be defined—the InvoiceApproval application in the case of this article. Operations must be defined, then grouped into tasks, which must be assigned to roles. Roles, in turn, are where you assign domain users and groups. Once this setup is in place, authorization checks, in the simple scenario, are done by checking whether one of the roles a principal is assigned to has been transitively granted the right to execute a task.

The sample includes a simple definition with four tasks and five roles (see Figure 5). Note that the Users Restricted by Business Hours not only has a set of users and groups assigned to it, but also has an associated business rule, which means that the users and groups are members of the role only if the business rule evaluates to True. (This particular business rule simply checks whether the time of day is between 9:00 AM and 5:00 PM.)

As you may imagine at this point, the implication of inserting AzMan is that you may dynamically assign users and groups to tasks, create new roles, and define new business rules. In short, you have achieved an extra degree of freedom while at the same time postponing authorization to be determined at the time of operation as opposed to at development time.

Making Authorization Claims and Enabling the Client

At this point, the entire infrastructure is basically ready. The pipeline and related functionality have been put in place and Authorization Manager is set up. All I need now is a way to specify the authorization claims at the service and to set up the Web service proxy at the client side to send along the required headers.

As described earlier, the WSE 3.0 authorization assertion I've developed is configured entirely in the policy file, as shown in Figure 3, using claims like:

<action
  name="http://msdn.microsoft.com/msdnmagazine/invoices/ViewInvoices"
  tasks="ViewInvoices" />

Note, though, that the tasks attribute may include more than one task. For example:

tasks="Task1;Task2"
With this syntax, I indicate that the sender should be authorized for at least one of these tasks. This purely declarative configuration approach enables authorization to be inserted into the WSE pipeline at any time after the service has actually been deployed, as long as the client has been configured or programmed to supply their identity as a KerberosToken in the WS-Security SOAP header. The developer of the service is therefore relieved of making the right authorization checks imperatively in code (for example, by using the IsInRole method of IPrincipal), as long as a list of endpoints is supplied for operations to configure as part of policy setup and maintenance.

At the client side, you simply need to make sure to use the WSE 3.0 proxy and then attach a Kerberos token for constructing the message according to the policy file at the client side. This file is similar to that on the service side, except no authorization assertions are present on this side of the message exchange.

The code in Figure 10 shows the code that is necessary for putting together a message with the format required by the service side so it can both authenticate and authorize the sender. Note how the policy is set and the Kerberos token is attached. After doing this, you're ready to send messages.

Figure 10 Setting Up the WSE Proxy at the Client Side (Close Figure 10)
private SecureInvoiceServiceWse GetProxy()
{
    SecureInvoiceServiceWse proxy = new SecureInvoiceServiceWse();

    // Specify the policy to apply
    proxy.SetPolicy( "KerberosPolicy" );

    string hostName = proxy.Url.IndexOf( "localhost" ) >= 0 ?
        System.Net.Dns.GetHostName() : new Uri( proxy.Url ).Host;

    KerberosToken kerberosToken =
        new KerberosToken( "host/" + hostName );

    proxy.SetClientCredential( kerberosToken );

    return proxy;
}

The Bigger Picture

Software engineering is all about platforms, languages, and tools. Applying an engineering approach to service infrastructure and security within the domain of established architectures requires evaluating the existing nuts and bolts that are available to provide a solution. You cannot develop robust and secure services without thoroughly understanding the operating system, the hosting options, and the tools. In my experience, whenever you consider "rolling your own," you either don't fully understand the thinking behind your platform or you have a very compelling cost/benefit ratio for the work that needs to be done.

Apart from the .NET Framework 2.0 and WSE 3.0 being Beta technologies, there are two particular areas that stand out as needing work beyond what is presented here.

Enterprise Library AzMan Provider The AzMan provider that is delivered with Enterprise Library works fine, but is limited in support for more advanced AzMan capabilities. The challenge is to write a new provider with full AzMan support without breaking the Enterprise Library IAuthorizationProvider interface. Thread-safe caching and pooling of open connections and contexts could also be considered for reducing the overhead that is incurred by the authorization step.

Authorization Manager AzMan is a feature of Windows Server 2003 definitely worth your consideration, but be prepared to address some staging and deployment issues. Also, if you work with custom SIDs that don't exist within Active Directory, which may be the case for an Internet solution, you may run into performance issues as the number of SIDs inside the AzMan store increases.

With the .NET Framework 2.0 and WSE 3.0, if you keep with the good engineering practices related to the trusted subsystem approach, and if you generally work along the lines of the platform, you can take one more small step towards securing your services without giving up flexibility in your production environment.

Niels Flensted-Jensen, a software architect and engineer, is a founder of CI Networks, a Danish software consultancy and component developer. Niels focuses on service-and process-oriented enterprise development centered on the Microsoft platforms. Reach him at blog.flensted-jensen.com.


How Do They Do It?: A Look Inside the Security Development Lifecycle at Microsoft

How Do They Do It?

A Look Inside the Security Development Lifecycle at Microsoft

Michael Howard
This article discusses:
  • Overview of the Security Development Lifecycle
  • Security in the design and development processes
  • Threat modeling and testing
  • Security reviews and responses
This article uses the following technologies:
Security Development Lifecycle
Contents
Leadership and Education
The Design Phase
Threat Modeling
The Development Phase
Security Testing
Starting a Security Push
Final Security Reviews
The Security Response
Does SDL Work?

T
he goals of the Security Development Lifecycle (SDL), now embraced by Microsoft, are twofold: to reduce the number of security-related design and coding defects, and to reduce the severity of any defects that are left. This follows our oft-cited motto, "Secure by Design, Secure by Default, Secure in Deployment and Communication" (also known as SD3+C). SDL focuses mainly on the first two elements of this motto. Secure by Design means getting the design and code secure from the outset, and Secure by Default is a recognition that you never will. To be realistic, you never will get the code 100 percent correct, but more on this later when I discuss attack surface reduction.

This article outlines how to apply the SDL to your own software development processes. I will explain how you can take some of the lessons we have learned at Microsoft when implementing SDL, so you can use these concepts in your own development process. But before I get started, I want to make clear that SDL is process-agnostic as far as how you go about developing software. Whether you use a waterfall model, a spiral model, or an agile model, it really doesn't matter; you can still use the process improvements that come from SDL. SDL involves modifying a software development organization's processes by integrating measures that lead to improved software security. The really great news is SDL does improve software quality by reducing security defects.

SDL adds security-specific checks and measures to any existing software development process. Figure 1 shows how SDL maps onto a "generic" process. If it makes you happy, wrap the SDL around a spiral or down a waterfall.

Figure 1 Mapping SDL onto a Generic Process (Close Figure 1)

I'll take a look at each major phase and outline what you can do within your own organization to implement SDL.

Leadership and Education

I'm often asked why SDL has been so successful at Microsoft. The answer is very simple: executive support, and education, and awareness. Getting Bill Gates and Steve Ballmer committed to SDL was critical, but just as critical is an educated engineering workforce.

For leadership, you need to nominate one or more individuals to be the point people for security. Their jobs include staying on top of security issues, pushing the security practices on the development organization and being the voice of reason when it comes to making tough security decisions. (If you're reading this, that person is probably you.) The leadership person or people should monitor the various security related newsgroups, such as Bugtraq (www.securityfocus.com).

If your engineers know nothing about the basic security tenets, common security defect types, basic secure design, or security testing, there really is no reasonable chance they could produce secure software. I say this because, on the average, software engineers don't pay enough attention to security. They may know quite a lot about security features, but they need to have a better understanding of what it takes to build and deliver secure features. It's unfortunate that the term security can imply both meanings, because these are two very different security realms. Security features looks at how stuff works, for example the inner operations of the Java or common language runtime (CLR) sandbox, or how encryption algorithms such as DES or RSA work. While these are all interesting and useful topics, knowing that the DES encryption algorithm is a 16-round Feistel network isn't going to help people build more secure software. Knowing the limitations of DES, and the fact that its key size is woefully small for today's threats, is very useful, and this kind of detail is the core tenet of how to build secure features.

The real concern is that most schools, universities, and technical colleges teach security features, and not how to build secure software. This means there are legions of software engineers being churned out by these schools year after year who believe they know how to build secure software because they know how a firewall works. In short, you cannot rely on anyone you hire necessarily understanding how to build security defenses into your software unless you specifically ask about their background and knowledge on the subject.

Some good sources for online and instructor-led security education include Microsoft eLearning (www.microsoftelearning.com/security). The security guidance for developers is derived from some of the security basics material we present at Microsoft.

You should also build a library of good security books, such as those listed in Figure 2.

Figure 2 Recommended Security Books (Close Figure 2)

Book Title | Developers | Testers | Designers | Windows | *nix
Writing Secure Code, 2nd Ed by David LeBlanc and Michael Howard |  |  |  |  |
19 Deadly Sins of Software Security by Michael Howard, David LeBlanc, and John Viega |  |  |  |  |
Building Secure Software by John Viega and Gary McGraw |  |  |  |  |
Gray Hat Hacking by Shon Harris, et al. |  |  |  |  |
How to Break Software Security by James Whittaker and Herbert Thompson |  |  |  |  |

Some groups at Microsoft have set up book clubs where they each read a chapter of a given book and then discuss it with the group. To further their knowledge, they search for examples of defects or design issues from common security stomping grounds, like Bugtraq.

Consider having security folks within your company hold presentations on security topics, including common security code defects (buffer overflow, cross-site scripting, SQL injection, integer arithmetic, weak crypto, and so on), secure design, threat modeling, and security testing. To add more depth and relevance to the discussions, find defects in your own code and use them as examples for the rest of the developers.

Finally, engineering staff should be encouraged to update their skills at least once a year—whether on their own or by attending staff development events—and this should be tracked so people don't fall between the cracks. It's really important that people stay on top of the security landscape, as it changes rapidly. A common bug today may be a significant security vulnerability tomorrow.

The Design Phase

The best opportunity to influence the security design of a product is early in the product lifecycle. This is the phase in which architects, developers, and designers traditionally do the bulk of the feature design. Design typically begins with a very short description of the intended functionality (sometimes in the form of a short functional summary), followed by a functional specification that describes the customer's view of the feature, which is, in turn, followed by a design specification that describes the technical details outlining how the feature will be implemented.

If you're using an agile development method, you may just opt for the shorter one-page summary, but this page should include the security aspects of the component. Functional specifications may need to describe security features that will be directly exposed to customers, such as requiring end user authentication to access specific data or advanced functionality. Design specifications will need to describe how to implement security features, as well as to ensure that all functionality is implemented as secure features. Notice the use of the word secure rather than security. Secure features are defined as ensuring that all functionality is well engineered with respect to security, such as robust use of Crypto APIs, use of managed code wherever possible, rigorously validating all data before processing it, and several other considerations. When designing features, it is crucial to carefully consider security concerns in order to avoid trying to bolt security onto the product near the end of the design process.

All functional and design specifications, regardless of document size, should contain a section describing how the component impacts security. To get some ideas on what to add to this section, you should review RFC 3552 "Guidelines for Writing RFC Text on Security Considerations".

An important part of the design process is to understand how you will reduce the attack surface of your application or component. Anonymously accessible and open UDP ports to the Internet represent a larger attack surface than, say, an open TCP port accessible only to a restricted set of IP addresses. I don't want to spend too much time on the subject here; instead you should refer to my article on attack surface reduction "Mitigate Security Risks by Minimizing the Code You Expose to Untrusted Users" in MSDN®Magazine November 2004. Figure 3 should give you a quick reference for reducing the attack surface of your code.

Figure 3 Attack Surface Reduction (Close Figure 3)

Higher Attack Surface | Lower Attack Surface
Code executing by default | Code off by default
Open socket | Closed socket
UDP | TCP
Anonymous Access | User Access
Internet Access | Local Subnet Access
Running as SYSTEM or admin | Not running as SYSTEM or admin!
Weak ACLs | Strong ACLs

Threat Modeling

Threat modeling must be completed during the product design process. A team cannot build a secure product unless it understands the assets the product is trying to protect (customers' personal information such as credit card numbers, not to mention their computers), the threats and vulnerabilities introduced by the product, and details of how the product will mitigate those threats. Additionally, it is also important to consider threats and vulnerabilities present in the environment in which the product is deployed or those that arise due to interaction and interfacing with other products or systems in end-to-end real world solutions. To this end, the design phase of a product cannot be considered complete until a threat model is in place. Threat models are critical components of the design phase and will reference both a product's functional and design specifications to describe both vulnerabilities and mitigations.

Understanding the threats to your software is a critical step to creating a secure product. Too many people bolt security technology to their app and declare it secure, but the code is not secure unless the security countermeasures are really resolving real-world threats. That's the goal of threat modeling. To get a good feel for the process in less than 30 minutes, I would recommend you read the blog entry "Guerrilla Threat Modeling" written by my colleague Peter Torr.

The Development Phase

During the development phase you should implement security tools, security checklists, and secure coding best practice to help implement a secure design. Remember, a secure design could quite easily be rendered insecure by a weak implementation.

Before I get started on this, I want to point out something very important. Security tools will not make your software secure. They will help, but tools alone do not make code resilient to attack. There is simply no replacement for having a knowledgeable work force that will use the tools to enforce policy. The new version of Visual Studio® 2005 Team System Developer's Edition includes some very, very useful security tools:

PREfast PREfast is a static analysis tool for C/C++ code. It can find some pretty subtle security defects, and some egregious bugs, too. This is lint on security steroids.

Standard Annotation Language (SAL) Of all the tools we have added to Visual Studio 2005, this is the technology that excites me the most because it can help find some hard to spot bugs. Imagine you have a function like this:

void *function(
    char *buffer,
    DWORD cbBufferLength);

You know that buffer and dwBufferLength are tied at the hip; buffer is cbBufferLength bytes long. But the compiler does not know that—all it sees is a pointer and a 32-bit unsigned integer. Using SAL, you can link the two. So the header that includes this function prototype might look like the following:

void *function(
    _in_bytecount(cbBufferLength) char *buffer,
    DWORD cbBufferLength);
Please note the final syntax used for SAL may change before Visual Studio 2005 ships.

FxCop You may already know of FxCop—it's a tool to find defects, including security defects in managed code. It's available as a download from www.gotdotnet.com, but the version in Visual Studio 2005 is fully integrated, and includes some new issues to watch out for.

Application Verifier AppVerifier is a runtime tool that operates on a running application. It can be used to trap memory-related issues at run time, including heap-based buffer overruns.

Other tools and requirement at Microsoft include:

  • All unmanaged C/C++ code must be compiled with the /GS stack overrun detection capability.
  • All unmanaged C/C++ code must be linked using the /SafeSEH option.
  • All RPC code must be compiled with the MIDL /robust flag.
  • Security issues flagged by FxCop and PREfast must be fixed.
  • The functions shown in Figure 4 are banned for new code, and should be removed over time for legacy code.

Figure 4 Sample Banned Functions (Close Figure 4)

Banned API | Strsafe Replacement | Safe C and C++ Libraries
strcpy, wcscpy, _tcscpy, _mbscpy, lstrcpy, lstrcpyA, lstrcpyW, strcpyA, strcpyW | String*Copy or String*CopyEx | strcpy_s
strcat, wcscat | String*Cat or String*CatEx | strcat_s
wnsprintf, wnsprintfA, wnsprintfW | String*Printf or String*PrintfEx | sprintf_s
_snwprintf, _snprintf | String*Printf or String*PrintfEx | _snprintf_s or _snwprintf_s
wvsprintf, wvsprintfA, wvsprintfW, vsprintf | String*VPrintf or String*VPrintfEx | _vstprintf_s
_vsnprintf, _vsnwprintf | String*VPrintf or String*VPrintfEx | vsntprintf_s
strncpy, wcsncpy | String*CopyN or String*CopyNEx | strncpy_s
strncat, wcsncat | String*CatN or String*CatNEx | strncat_s
scanf, wscanf | None | sscanf_s
strlen, wcslen, _mbslen, _mbstrlen | String*Length | strlen_s

You can read about the Strsafe string replacement code in "Strsafe.h: Safer String Handling in C". The Safe C library is the new C runtime library replacement built into Visual Studio 2005. You can read about it at "Safe! Repel Attacks on Your Code with the Visual Studio 2005 Safe C and C++ Libraries".

Security Testing

A very useful testing technique for finding security defects is "fuzzing," which means taking valid data, morphing that data, and then observing an application that consumes the data. In its simplest form, you could build a library of valid files that your application consumes, and then use a tool to systematically corrupt a file and have your application play or render the file. Run the application under application verifier with heap checking enabled to help uncover more errors. Examples of morphing data include:

  • Exchanging random bytes in a file
  • Writing a random series of bytes of a random size at a random location in file
  • Look for known integers to change the sign, or make them too large or too small
  • Look for ASCII or Unicode characters and set the trailing NULL character to be non-NULL

Michael Sutton and Adam Greene gave an interesting session at Blackhat USA 2005 about fuzzing. You can read it at The Art of File Format Fuzzing. Ejovi Nuwere and Mikko Varpiola also gave an interesting presentation on fuzzing the VoIP networking protocol, available at The Art of SIP Fuzzing and Vulnerabilities Found in VoIP.

Starting a Security Push

A security push is a team-wide focus on threat model updates, code review, testing, and documentation scrub. Note that the push is not a quick fix for a process that lacks security discipline; rather, it is a concerted effort to confirm the validity of the information in your Security Architecture documentation, to uncover changes that may have occurred during the development process, and to identify and remediate any remaining security vulnerabilities you may discover. It is not possible to build security into software with just a security push alone.

There is no easy way to determine the length of time needed for a security push; the push duration is ultimately determined by the amount of code that needs to be reviewed for security, as all pushes to date have been gated by code quantity. Teams are strongly encouraged to try to conduct security code reviews throughout the development process, once the code is fairly stable, as the quality of code reviews will suffer from trying to condense too many code reviews into too short of a time period.

The rule of thumb is to determine critical code, using heuristics such as exposure to the Internet, handling sensitive or personally identifiable information and so on, and mark that as priority one code. That code must be reviewed during the push, and the push cannot be complete until that code is reviewed. Assign a name to every code file, and assign a name and priority to the code before the push.

Final Security Reviews

Once the end of the project draws near, a very important question must be answered: from a security viewpoint, is the software ready to ship? The Final Security Review (FSR) answers this question. It's performed by the central security team with help from the product team, but not by the product team alone. It's usually done a few months before the software is complete and includes:

  • Completing a questionnaire, which helps the security team focus its effort. Questions might include:
  • Do you have any ActiveX® controls marked Safe for Scripting?

List all the protocols you fuzzed.

  • Does your component listen on unauthenticated connections?
  • Does your component use UDP?
  • Does your component use an ISAPI application or filter?
  • Does any of your code run in System context? If yes, why?
  • Do you set ACLs in setup code?
  • Review bugs that are deemed "won't fix" and make sure they are not mismarked security defects.
  • Analyze security defects from other versions of the product and even competitors' products to make sure you have addressed all these issues. One common question we ask is, "how have you mitigated this broad category of security issues?"
  • Perform penetration testing of high-risk components, perhaps by a third-party company.

At the end of the FSR process, the findings are written up and a decision is made about releasing the software or reworking sections.

The Security Response

There are two major parts to this process. One is responding to security defects and working with people who find security issues in your code. The other aspect is learning from these mistakes. At Microsoft, we have dedicated staff whose job it is to perform root-cause analysis of defects. These documents are then used to influence future iterations of the SDL. Each postmortem doc includes:

  • Name and version of file.
  • Was this a design issue?
  • Was this a coding issue?
  • Source code diff if it's a coding bug.
  • Does the bug affect other versions of the product? If yes, why? If not, why not? (They are both very valid questions.)
  • What tests could have found this defect?
  • What tools could have found this defect?
  • Do we need to change our education, tools, or process to find these issues?

The answers are then used to modify SDL. The SDL at Microsoft is updated twice a year, January 1 and July 1.

Does SDL Work?

So the big question is "does SDL work? Does the employment of Security Development Lifecycle techniques result in software that is more secure?" The answer is a resounding Yes! We have seen the number of security defects be reduced by approximately 50 to 60 percent when we follow SDL. The simple fact is that every product touched by SDL has fewer security defects. Period. And that certainly makes it worth pursuing.

Michael Howard is a senior security program manager at Microsoft focusing on secure process improvement and best practice. He is the coauthor of 19 Deadly Sins of Software Security (McGraw-Hill Osborne, 2005) and Processes to Produce Secure Software (Dept. of Homeland Security National Cyber Security Division).


Editor's Note: Many Levels of Security

Editor's Note
Many Levels of Security

Every year at this time, we bring you our now-famous security issue. We recognize the vast importance of writing and deploying secure code—it affects so many areas of concern—which is why we devote an entire issue each year to the topic. This 2005 security issue marks the fourth year that we have gathered the most relevant coverage of current best security practices for programmers who want to keep their code safe.

Of course, we don't save up all this security information for a single month's issue. That wouldn't be fair. All throughout the year, every year, we publish articles and columns on security to make sure that you have the best information possible to make your code more impervious to attack and your systems safer from hackers, accidents, and all sorts of harm. In fact, security is such a big deal to us that we've gone beyond mere print and produced a special companion security CD. In it we've gathered the best, most relevant security coverage we've published over the past five years of MSDN Magazine (see our Web site for more info).

We've covered quite a lot of ground on the topic of security, but it's never too much. It's important to remember how broad a topic this really is. The stuff that most programmers hear about—firewalls, virus checkers, and sneaky Web attacks—is really only scratching the surface. We know you also worry about the next level down: how do you fortify your code to minimize the possibility that a malicious attacker will use your program as the springboard into your machine, your network, or your company's confidential data?

This month's articles will put you on the right path in that respect. We present tips on code access security in the .NET Framework 2.0. We discuss best practices that should be followed when building secure Web applications with ASP.NET 2.0 and IIS, and we look at how the provider model and new Membership API can be used to port your existing authentication code to ASP.NET 2.0. We show you the latest sandboxing techniques to help you improve security when hosting untrusted add-ins. And we make many other useful security suggestions that you'll want to read now and refer to later as well.

Security is not just a matter of making code safer either; the physical security of your data is equally important. As a programmer, do you have backups of your code in an alternate location? If you're a photographer, do you have all your best images stored on a second machine far away from your home? Of course, as you know, all the programming tips in the world won't protect your assets if you're not prepared for the worst possible occurrences, but following some of the best security practices in your own environment is a good way to start.

On that note, it goes without saying that we are all saddened by the destruction and devastation caused by Hurricane Katrina. Our hearts go out to the thousands who have suffered, and our gratitude to the thousands who gave up their personal comfort and safety to offer their help. We all have friends, family, or acquaintances who were affected by this tragedy. Although we may not know you all personally, we have many readers in the Gulf Coast region, some of whom surely have suffered great losses. We will keep you all in our thoughts.

Many inspiring stories of help and heroism have followed this disaster. We send out our thanks to people like John Morello (who writes for our sister publication, TechNet Magazine) who went to New Orleans to help set up basic IT services and networks at shelters throughout the area. A small comfort maybe, but these services enabled people who were unable to get back to their homes to begin to initiate contact with their missing friends and relatives. It never ceases to amaze us how generous people can be at times like this. Thanks, and take care.

—J.T.

Thanks to the following Microsoft technical experts for their help with this issue: Kawarjit Bedi, Eric Bidstrup, Arjun Bijanki, Eugene Bobukh, Brandon Bray, Simon Calvert, Lakshan Fernando, Mark Fussell, Matthew Gibbs, Jim Hogg, Tomasz Janczuk, Charlie Kaufman, Ronald Laeremans, Martyn Lovell, Ulzii Luvsanbat, Shahrokh Mortazavi, Matt Powell, Stefan Schackow, Sidd Shenoy, Payam Shodjai, Richard Turner, Scott Woodgate, and Ting-Hao Yang.

Active Directory, ActiveX, BizTalk, Excel, IntelliSense, JScript, Microsoft, MSDN, Visual Basic, Visual Studio, Windows, Win32, Windows NT, and Windows Server are either registered trademarks or trademarks of Microsoft Corporation. Other trademarks or tradenames mentioned herein are the property of their respective owners.

MSDN Magazine does not make any representation or warranty, express or implied with respect to any code or other information herein. MSDN Magazine disclaims any liability whatsoever for any use of such code or other information.


New Stuff: Resources for Your Developer Toolbox

New Stuff
Resources for Your Developer Toolbox
Marnie Hutcheson

Contents
Refactor Your C# Code
Add Original Toolbars to Your App
Use Image Recognition for Web Testing
The Bookshelf

Refactor Your C# Code

Code refactoring is defined as the act of changing code without changing what it does. The actual work involved in refactoring—extracting methods from oversized modules, tidying up and unifying variable names, identifying unhandled exceptions, and all the other cleanup, simplification, and standardization chores—can be daunting, indeed, and risky too. But when code gets too complex or when you plan to move your code to a new architecture, you really should consider refactoring.

C# Refactory by Xtreme Simplicity is a set of must-have code-refining tools for your C# solutions. C# Refactory has its own parsing engine and analysis, so it's not limited to the information available in Visual Studio® .NET. C# Refactory parses your entire solution, not just the current file. It's written in C# and fully integrated into Visual Studio .NET so that the appropriate refactory functions are always available to you in the right-click menu no matter what you're doing.

C# Refactory metrics show you statistics at the file, project, and solution level. You can explore individual file metrics, classes, and methods just by right-clicking on any line of source code in the editor. You can sort by any of the metrics and identify methods and classes that have become too complicated. Class metrics show all your classes along with the number of lines of code, comments, cyclomatic complexity, and so on.

Once you have found those long, complex methods, you can use the Extract Method feature to pull out individual methods—just highlight a section of code and extract it. C# Refactory creates a new method along with the required input and output parameters. It also accounts for looping and early exits.

Price: $99 per single license.

www.xtreme-simplicity.net

Add Original Toolbars to Your App

DotNetBar Suite 5.0 by DevComponents.com is a set of 12 integrated components that let you create unique user interfaces in Visual Studio .NET. One of the most interesting additions in this version of DotNetBar is a BubbleBar control that solves the problem of small, unreadable, hard-to-click toolbar buttons. BubbleBar creates a UI experience similar to the one provided by the Mac OS X interface. When the mouse hovers over the toolbar, the image under the mouse will expand smoothly to a predefined size, so users can see the button and have a larger area to click on. Improved designers and other unique controls, like Dockable auto-hide and windows with Diamond Docking Hints guides, make the job of creating your interface easier. There are lots of helpful tutorials on the site to get you started.

Price: $199 per single license. www.devcomponents.com

Use Image Recognition for Web Testing

The Internet Macros Image Recognition Plugin by iOpus is a new add-on for both the iOpus PRO Edition and Scripting Edition products—two solutions for Web automation and testing.

The Image Recognition Plugin uses machine vision technology for testing Web sites by comparing actual images instead of X/Y coordinates. It recognizes and compares images rendered in the browser, regardless of the technology that created them.

The plug-in gives you a new image search function, a command-line interface, batch file support, and the ability to perform Web site response time measurements. The plug-in can switch between different Web pages automatically with its tabbed browser interface.

The Scripting Edition is designed for use by Web professionals and people responsible for Web site testing. It includes all of the functionality of the PRO Edition, plus an easy-to-use scripting interface that extends the popular Windows® Scripting Host with the ability to replay Internet Macros and extract Web page data. You can also use Visual Basic®, Microsoft Excel®, Access, Visual Basic .NET, C#, and Perl to access this interface.

Price: Image Recognition Plugin for PRO and Scripting Editions is $998.

www.iOpus.com

The Bookshelf

UML 2.0 in a Nutshell by Dan Pilone and Neil Pitman, (O'Reilly Media, Inc.) is a comprehensive guide to the syntax and semantics of the Unified Modeling Language version 2.0. It covers the major diagram types, including class diagrams, use case diagrams, composite structures, and interaction diagrams. Each chapter has detailed explanations of all the symbols, notations, and line types.

On the surface, UML is a visual language for capturing software designs and patterns, but if you dig a little deeper you'll find that UML can be applied to quite a few different areas and can capture and communicate everything from company organization to business processes all the way to distributed enterprise software.

For that reason, the book is aimed at a wide audience. Pilone and Pitman discuss modeling business organizations, real-time systems, and databases, as well as the customary software development modeling techniques.

Price: $34.95, 240 pages.

www.oreilly.com

Data Structures and Algorithms Using Visual Basic.NET, by Michael McMillan (Cambridge University Press), is a dedicated reference for Visual Basic .NET. It gives you a comprehensive discussion of the major data structures and algorithms in Visual Basic, without having to translate them from C++ or the Java language. The book covers arrays, linked lists, hash tables, dictionaries, trees, graphs, sorting, and searching, as well as more advanced algorithms, such as probabilistic algorithms and dynamic programming— all in an object-oriented fashion.

The book includes a tutorial on how to use data structures and algorithms and a reference for implementation using Visual Basic .NET for data structures and algorithms from the .NET Framework Class Library, as well as those that you will need to write for yourself.

The author's approach is very practical. For example, he uses timing tests rather than Big O analysis to compare the performance of data structures and algorithms. This book can be used in both beginning and advanced computer programming courses that use the Visual Basic .NET language, but it is also useful for the practitioner.

Price: $45, 412 pages.

www.cambridge.org

.NET Gotchas (O'Reilly Media, Inc.), by Venkat Subramaniam, is a valuable resource for programmers. Each pitfall is fully explained in the book, using Visual Basic .NET and C# examples to show you how the unsuspecting programmer can inadvertently fall victim to these code demons, and how these problems manifest. There are comparisons between C++ and C#, and Visual Basic 6.0 and Visual Basic .NET throughout the book that are really helpful if you are migrating.

The main topics include snags and pitfalls when working with types, important compiler warnings, and other areas of concern. The section on garbage collection goes a long way to explaining how some mysterious performance glitches can happen if you rely on finalizers, and how inheritance and polymorphism signature mismatches can lead to method hiding. The multithreading section shows you how exceptions thrown from threads in the pool in the .NET Framework 1.x are lost, and the section on COM interop and enterprise services discusses inconsistencies across languages.

Price:$39.95, 396 pages.

www.oreilly.com

Send your New Stuff to newstuff@microsoft.com.

All prices were confirmed at press time and are subject to change.

Marnie Hutcheson is president of Internet Development Associates (Ideva), a firm in Ocala, Florida that specializes in Internet and intranet Web application design and development. She has published technical papers and books on various computing topics. You can reach her at marnie@ideva.com.


Web Q&amp;A: ASP.NET Session State, Validation, DataGrids, and More

Web Q&A
ASP.NET Session State, Validation, DataGrids, and More
Edited by Nancy Michell

Q If I have an ASP.NET application using session state and I launch a new browser window from one of my pages, is there a way to force a new session? I need to have two unique sessions for a user, one for the original browser window and one for the new browser window.

A If you are using a cookie-based session and you just open a new window inside the same Microsoft® Internet Explorer process, you'll share cookies, including the SessionID cookie and thus the same session. If you launch this new window in a separate Internet Explorer process, then you will have a new session even for the same application.

If you're using cookieless session state, the server appends the ID to the URL. According to Knowledge Base article 307598, "INFO: ASP.NET State Management Overview,": "When a session is initiated on first request, the server issues a unique session ID to the user. To persist the session ID, store it in an in-memory cookie (which is the default), or embed it within the request URL after the application name. To switch between cookie and cookieless session state, set the value of the cookieless parameter in the Web.config file to true or false. In cookieless mode, the server automatically inserts the session ID in the relative URLs only. An absolute URL is not modified, even if it points to the same ASP.NET application, which can cause the loss of session variables."

Therefore, you could accomplish what you're looking for by launching a new window using an absolute URL to the same app.

Q I have an ASP.NET page with multiple validator controls. Is it possible to control the order in which the validation rules are executed, client side and server side? Do I need to somehow add an "If field is not null" to the validation to avoid a runtime error? For example, if Field A is required and used in a formula for another field's validator (C > A*B) would that generate an error if the formula happened to be evaluated first and Field A happened to be empty?

A You cannot control the order of the validation rules, but if you have a RequiredFieldValidator and a RegularExpressionValidator, the RequiredFieldValidator must fire first before it can check if the data in the control matches the expression.

With regard to validation controls, if you set a RequiredFieldValidator on Field A and a RegularExpressionValidator on the same field, then the RequiredFieldValidator will execute first. But in your scenario, the order of the fields will be important depending on when the validation takes place. If it happens after tabbing out of the control, then the order is definitely important. Plus, client-side validations will execute before the server-side validation.

The firing will happen automatically for all controls with a validation in place, whenever a postback is triggered. The RequiredFieldValidator goes first as no other validation can be done without data in the control.

You do not need additional code to fire a validator or suppress it. The beauty of ASP.NET is that it handles these for you. You can, however, cancel the validation and move on by adding additional code in your application. This is because the validation occurs before the server processes the page and a user could get trapped by validation unless you have a way to cancel the validation without posting back.

Client-side validation is handled by JScript® and will work only on Internet Explorer 4.0 or later, as they support the Document Object Model required to perform client-side validation.

Q My ASP.NET app generates pages dynamically. I don't want the pages cached, so I've added the following code:

Response.Cache.SetCacheability(System.Web.HttpCacheability.NoCache);
This sets the Cache-Control: no-cache header. However, how can I cache static .css and .png files?

A It all depends on whether you want the caching to occur in usermode or you want to take it to the next level and cache in the kernel (http.sys). Either way, you should look at the OutputCache info at @ OutputCache. There is also a TechNet IIS Insider article about kernel mode caching and ASP.NET that might be of interest. You can read that at Troubleshooting ASP.NET Page Output Caching.

One solution is to put all static content in a folder (or folders) separate from the active content folders. In IIS, you should explicitly set the time-to-live (TTL) expiration for these folders to a reasonable amount of time—for example, 12 hours. You'll get good performance improvement.

Q I'm getting strange error messages from time to time in my ASP.NET 2.0 Beta 2 application:

Error Message: This is an invalid webresource request.
I have no references to Webresource.axd in my code. I have an error logger in Global.asax that sends me an e-mail when an error occurs, but I don't know what the user did to trigger this. What could cause it?

A You may have an ill-formed WebResource URL. There should be a couple of query string parameters following every WebResource URL. If that's not the case, then your code may be emitting invalid requests. ASP.NET uses WebResource.axd too, so there may be requests to it even if you don't have code that uses it. Or, you might be using a third-party control that uses this feature.

If you can reproduce this, you could take a look at the rendered source of the page that when requested throws the error, and check every WebResource URL to see if any are invalid. Taking a look at IIS logs may also shed some light on the subject. Read the MSDN® online article by Victor Garcia Aprea at Handling Client Files in ASP.NET Whidbey.

Also you might run into issues if you have a Web farm and the machine keys are not identical. Note that if you're using ASP.NET 2.0 Beta 2, it might be double-encoding URLs that reference WebResource.axd, which would obviously generate invalid URLs. This problem will be fixed in the final version of ASP.NET 2.0.

Q I have a DataGrid linked to SQL Server™ in which the first column is a checkbox. The user can select multiple values in the grid. When he retrieves this list, how can I make the page indicate which items were originally checkmarked? Here's a sample:

<asp:TemplateColumn>
    <ItemTemplate>
        <asp:CheckBox id="chkThis" DataField="Choice"
                      SortExpression="Choice" runat="server"/>
    </ItemTemplate>
</asp:TemplateColumn>

A To get the functionality you're looking for, your TemplateColumn should look like this:

<asp:TemplateColumn SortExpression="Choice">
    <ItemTemplate>
        <asp:CheckBox runat=server Checked='<%#
            DataBinder.Eval(Container.DataItem, "Choice") %>' />
    </ItemTemplate>
</asp:TemplateColumn>
When your DataGrid is sorted by the Choice field, you should handle the OnSort event, sort your resultset, and rebind the grid.

Q I have a DataGrid with a templated column that hosts a DropDownList control. How do I get the underlying grid row from within the dropdown's SelectedIndexChanged event?

A To get to the DataGrid's row, you need to traverse up the control tree. The DropDownList is contained inside a TableCell, which is contained inside the TableRow you're looking for. The TableRow is strongly typed as a DataGridItem, which inherits from TableRow. To get to the DataGridItem, you need to get the parent (the DataGridItem) of the parent (the TableCell) of the DropDownList control, as shown here:

DataGridItem gridItem = (DataGridItem)ddl.Parent.Parent;
A more general approach would be to use:
DataGridItem gridItem = (DataGridItem)ddl.Container;
This will work no matter how deep in your control tree your sender control is, as long as it's not inside another INamingContainer.

Q I have a situation that demands that a DataGrid (or some other data viewer) opens in updatable mode with all the rows updatable at the same time. I want some of them to be textboxes, some dropdowns, and some non-updatable columns. I know how to do it using an edit column in a DataGrid, but how can I do this for multiple rows?

A Set up template columns and in the ItemTemplate, add the editable controls you want to use. You'll need to add a control for an explicit user gesture to perform the save. An example is documented here: Top Questions about the DataGrid Web Server Control.

Q What is the best practice for persisting a recordset in ASP.NET? I have a query that, depending upon which items are selected in the dropdown lists, returns specific data to a DataGrid which the user can then page through.

A The best way to deal with multiple queries to the same data is to cache the DataSet in the ASP.NET cache. You can set the cache expiration policy so that it's appropriate for your data (short expiration if your data changes a lot, long or non-expiring if your data rarely changes). Then when you need to bind your data to the DataGrid, you can retrieve the DataSet from the cache if it's present (memory pressure can expire data from cache) or requery your data if it's not present. You can see a good example under "Cache arbitrary objects in server memory".

Q I am using EditCommandColumn in a DataGrid and I have created an event handler that handles the update event. I have also added OnUpdateCommand="DataGrid1_Update" in the DataGrid tag in the HTML. I know that this event handler is executing on the update event, but the values I'm getting are the old values. Why is this happening?

A How are you data binding on page load? Make sure you're only binding the first time the page is loaded, like this:

Sub Page_Load(sender As Object, e As EventArgs)
    If Not IsPostBack Then BindGrid()
End Sub
When you click on the Update button, the page gets posted to the server. If you don't add the "Not IsPostBack" condition, the DataGrid gets re-bound to the original data every time the page is loaded, which overwrites the new values entered by the user.

Q I've heard that there are many changes in Web.config in ASP.NET 2.0. What are they?

A There are actually quite a few: for example, there's IntelliSense®, new sections for providers, and the connectionStrings section. There's the new Configuration API that can read and write both Web.config and Machine.config files that reside on the same server, as well as on different servers via DCOM (the ASP.NET admin site is written on top of these new APIs). Certain sections of the Web.config file can be protected (encrypted using either DPAPI or RSA algorithms). This can be done via the configuration APIs or the enhanced ASPNET_REGIIS tool. See Walkthrough: Encrypting Configuration Information Using Protected Configuration for more information.

Send your questions and comments to webqa@microsoft.com.

Thanks to the following Microsoft developers for their technical expertise: Jeff Abrams, Chris Adams, Victor Aprea, Gaurav Bansal, Matthew Barbour, Stephane Bouillon, Michael Bundschuh, Rob Cameron, David Corbin, Andrew Duthie, Sudeep Gopalakrishnan, Tim Heuer, Gurpal Hundal, Ajoy Krishnamoorthy, Eilon Lipton, Alexander Louie, Jakub Oleksy, Markus Oswald, Polita Paulus, Minh Pham, Mike Pope, G. Joel Reyes, Rahul Sachdeva, Mohamed Sharaf, Srinivasa Sivakumar, Olav Tollefsen, and Radomir Zaric.


Data Points: DataSet and DataTable in ADO.NET 2.0

Data Points
DataSet and DataTable in ADO.NET 2.0
John Papa

Contents
DataTable Enhancements
Lightweight and Fast Looping
Loading Data
LoadOption Enumerator
Changing RowState
0 to 60
Wrapping It Up

ADO.NET 2.0 sports some exciting enhancements to the core classes found in ADO.NET 1.x and introduces a variety of new classes, all of which promise to improve performance, flexibility, and efficiency. There have even been some major changes throughout the lifecycle of the pre-beta and beta versions of ADO.NET 2.0, such as the improvements to the new batch updating process. With the final release of ADO.NET 2.0 fast approaching and the feature set becoming more stable, it's time to take a closer look at what's in store.

This month I will begin by exploring the improvements to the DataSet and DataTable classes, what they mean to you, and when you might want to use them. Sometimes in ADO.NET 1.x, especially when working with large rowsets, you can experience performance problems. I'll discuss how some of these performance issues have been addressed in ADO.NET 2.0 by changes to the indexing engine. I'll review the features that have been added to the DataTable class, as well as the loading options through the new Load method and the new methods that change a row's state. In the next installment of this column, I will discuss other improvements, such as the ability to perform batch updates, compressing DataSets for transport using binary serialization, and more.

DataTable Enhancements

In ADO.NET 1.x the DataSet got all the glory leaving the DataTable in the shadows. This is not to say that the DataTable was not a useful class on its own. The DataTable is the container for rows and columns and could be considered the focal point of all disconnected data. However, the DataSet gets most of the press because it can contain DataRelation and the DataTable objects.

However useful, the DataTable does have some limitations in ADO.NET 1.x that the DataSet does not. For example, the DataSet exposes a Merge method that can merge two DataTable objects within a DataSet, but the DataTable itself does not expose a Merge method. So if you had a DataTable on its own (not contained within a DataSet) and you wanted to merge it with another DataTable object, you would have to first create a DataSet object, put the first DataTable into it, and invoke the DataSet.Merge method, as I did in Figure 1.

Figure 1 Merging Two DataTable Objects in ADO.NET 1.x (Close Figure 1)
string sqlAllCustomers = "SELECT * FROM Customers";
string cnStr =
  @"Data Source=.;Initial Catalog=northwind;Integrated Security=True";

using (SqlConnection cn = new SqlConnection(cnStr))
{
    cn.Open();
    SqlCommand cmd = new SqlCommand(sqlAllCustomers, cn);
    SqlDataAdapter adpt = new SqlDataAdapter(cmd);
    DataTable dtCust1 = new DataTable("Customers");
    adpt.Fill(dtCust1);
    dtCust1.PrimaryKey = new DataColumn[]{dtCust1.Columns["CustomerID"]};

    DataTable dtCust2 = dtCust1.Clone();
    DataRow row1 = dtCust2.NewRow();
    row1["CustomerID"] = "ALFKI";
    row1["CompanyName"] = "Some Company";
    dtCust2.Rows.Add(row1);

    DataRow row2 = dtCust2.NewRow();
    row2["CustomerID"] = "FOO";
    row2["CompanyName"] = "Some Other Company";
    dtCust2.Rows.Add(row2);

    DataSet ds = new DataSet("MySillyDataSet");
    ds.Tables.Add(dtCust1);
    ds.Merge(dtCust2);

    dgTest.DataSource = dtCust1;
}

It's not difficult by any means, but it is annoying. In ADO.NET 2.0, the DataTable object now has a Merge method so you can merge two DataTable objects, like so:

dtCust1.Merge(dtCust2);

Another inconvenience in ADO.NET 1.x is that you cannot perform the basic XML operations on a DataTable without first associating it with a DataSet. For example, if you want to write a DataTable to XML, you need to load the DataTable into a DataSet and use the DataSet's WriteXml method. However, in ADO.NET 2.0, the DataTable has a WriteXml method so the problem is solved. In addition to the WriteXml method, the DataTable in ADO.NET 2.0 also exposes the ReadXml, ReadXmlSchema, and WriteXmlSchema methods.

The DataSet also has some new methods and properties. In fact, both the DataSet and the DataTable now expose the RemotingFormat property as well as the Load and CreateDataReader methods. The RemotingFormat property is used to indicate whether to serialize the DataTable or DataSet in binary or XML format. The Load method can be used to load data into a DataTable or DataSet in a variety of ways, which I'll discuss shortly.

Lightweight and Fast Looping

The DataTable's CreateDataReader method (named GetDataReader in previous Beta versions) creates an instance of the ADO.NET 2.0 DataTableReader. A DataTableReader created using DataTable.CreateDataReader will expose the same rows and columns as the DataTable. When a DataTableReader is created from a DataSet or a DataTable's CreateDataReader method, the DataTableReader will contain all of the rows from the container object, with the exception of deleted rows.

The DataTableReader is a lighter weight object than the DataTable and, unlike the DataReader (SqlDataReader), the DataTableReader is disconnected. This is a great feature because you get the lightweight object that you can iterate through quickly (like the DataReader) and it is disconnected from any data source (unlike the DataReader). A DataTableReader can be thought of as an iterator over the underlying table's rows, similar to a foreach enumeration of the contents of the table. However, unlike enumerating a table's rows (which will cause an exception when a row is added or deleted from the collection during the enumeration), a DataTableReader is resilient to changes being made to the underlying table and will correctly position itself in light of any modifications that occur.

The following example shows how you can create a DataTableReader and bind it to a DataGridView:

using (SqlConnection cn = new SqlConnection(cnStr))
{
    SqlCommand cmd = new SqlCommand(sqlAllCustomers, cn);
    SqlDataAdapter adpt = new SqlDataAdapter(cmd);
    DataTable dtCustomers = new DataTable("Customers");
    adpt.Fill(dtCustomers);
    DataTableReader dtRdr = ds.CreateDataReader();
    dgvCustomers.DataSource = dtRdr;
}

The DataTableReader can only be traversed in a forward manner, just like the DataReader. Again, like the DataReader, you move to the first row by using the Read method of the DataTableReader. If the DataTableReader was created from a DataSet that contained multiple DataTables, the DataTableReader will contain multiple resultsets (one per DataTable). Each subsequent resultset can be accessed using the DataTableReader by invoking the NextResult method (again, similar to the DataReader).

Figure 2 shows how to create a DataTableReader from a DataSet that contains two DataTable objects. Since the DataSet has two DataTables, the DataTableReader will contain two resultsets. You can loop through both resultsets in the DataTableReader using the Read and NextResult methods, as shown in Figure 2. Keep in mind that the DataTableReader only moves forward. So if you want to access the DataTableReader twice, because you want to loop through it again, you'll have to reload it after reading through all of its records the first time.

Figure 2 Looping Through a DataTableReader (Close Figure 2)
using (SqlConnection cn = new SqlConnection(cnStr))
{
    // Create the Command and Adapter
    SqlCommand cmd = new SqlCommand(sqlAllCustomers, cn);
    SqlDataAdapter adpt = new SqlDataAdapter(cmd);

    // Create a DataTable and fill it
    DataTable dtCustomers = new DataTable("Customers");
    adpt.Fill(dtCustomers);

    DataSet ds = new DataSet();
    ds.Tables.Add(dtCustomers);
    adpt.SelectCommand = new SqlCommand("SELECT * FROM Orders", cn);
    adpt.Fill(ds, "Orders");

    // Create the DataTableReader (it is disconnected)
    using(DataTableReader dtRdr = dtCustomers.CreateDataReader())
    {
        do
        {
            Console.WriteLine("******************************");
            while (dtRdr.Read())
            {
                Console.WriteLine(dtRdr.GetValue(0).ToString());
            }
        }
        while (dtRdr.NextResult());
    }
}

In Figure 2 the DataSet creates a DataTableReader using the CreateDataReader method. The order in which the DataTable's resultsets are added to the DataTableReader is the same as the order in which they appear in the DataSet. If you want to specify the order of the DataTable's resultsets, there is an alternate overloaded CreateDataReader method you can use.

Loading Data

A DataTableReader can also be used to populate a DataTable or a DataSet. In fact, using the new Load method of a DataSet or a DataTable, you can pass in a DataTableReader or any reader class that implements the IDataReader interface. The following example assumes there is a DataTable called dt1 that contains a schema and some rows. It creates a DataTableReader from the DataTable called dt1 and then turns right around and loads a second DataTable (called dt2) with the same data:

DataTableReader dtRdr = dt1.CreateDataReader();
DataTable dt2 = new DataTable();
dt2.Load(dtRdr);

In ADO.NET 1.x you could populate a DataSet or DataTable using the DataAdapter's Fill method. Alternatively, you could populate a DataSet from XML using the DataSet's ReadXml method. The introduction of the Load method in ADO.NET 2.0 makes it possible to load a DataSet or a DataTable from a class that implements IDataReader (like a DataTableReader or the SqlDataReader). When using the Load method to load several rows you can turn off notifications, index maintenance, and constraint checking by first invoking the BeginLoadData method, and then turn them back on by invoking the EndLoadData method. These methods, which are available in ADO.NET 1.x as well, can make the loading of data faster since ADO.NET does not have to stop after each row and recompute its indices, invoke notifications, or check constraints. Just make sure you remember to turn these things back on.

LoadOption Enumerator

The Load and the Fill methods have an overload that accepts one of the values from the LoadOptions enumeration. These settings introduce powerful new abilities which can be used to indicate how rows should replace existing rows during a Fill or Load operation into a DataSet or DataTable. This process assumes that a primary key has been set since it uses the primary key to determine how to replace or append the rows. These enumerator values help determine whether or not the current value and/or the original value get overwritten with the incoming rows' values. Figure 3 shows the three options and brief descriptions of each.

Figure 3 LoadOption Enumerator Settings (Close Figure 3)

Setting | Description
PreserveChanges | This is the default setting. Keeps the current values. Overwrites the original values with the incoming rows.
Upsert | Overwrites the current values with the incoming rows. Keeps the original values.
OverwriteChanges | Overwrites the current values with the incoming rows. Overwrites the original values with the incoming rows.

Each of these options has its place in an application given an appropriate circumstance. The OverwriteChanges option works well if you have a DataTable that contains data but you want to get any changed values that may exist in the database. Using this option will overwrite both the original and the current values in the DataTable with the values that come from the database. The key here is that when using OverwriteChanges, any data that has been modified (original version or current version) in the first DataTable will be overwritten while new rows will be added.

DataSet columns store an original and a current value. PreserveChanges will keep the current value intact while overwriting the original value. Upsert does the opposite of this as it keeps the original value intact while overwriting the current value.

Here is an example of when PreserveChanges might come in handy. Suppose a user named Peggy has opened a screen and loaded a DataGrid with customers from a DataSet. Peggy modifies the city of the customer with CustomerID ALFKI from Berlin to New York, but doesn't click the Save button. She then goes off for a cup of coffee. Meanwhile, Katherine modifies the same customer's city from Berlin to Miami. You'll now have a data concurrency issue if Peggy comes back from her break and saves the record. So in this situation, the original value for Peggy's customer record was Berlin and since she changed it to New York the current value is New York. Meanwhile, in the database the city is now Miami. If you want to reset the original values of Peggy's DataSet to what is in the database, you could get the data from the database into a DataTableReader and then load it into the DataSet using LoadOptions.PreserveChanges. Figure 4 illustrates how this works.

Figure 4 Testing LoadOptions (Close Figure 4)
using (SqlConnection cn = new SqlConnection(cnStr))
{
    SqlCommand cmd = new SqlCommand(sqlAllCustomers, cn);
    SqlDataAdapter adpt = new SqlDataAdapter(cmd);
    DataTable dtCustomers = new DataTable("Customers");
    adpt.Fill(dtCustomers);
    dtCustomers.PrimaryKey = new DataColumn[] {
        dtCustomers.Columns["CustomerID"] };

    // Pause here to execute this SQL directly against the database:
    // UPDATE Customers SET City = 'Miami' WHERE CustomerID = 'ALFKI'
    System.Diagnostics.Debugger.Break();

    // Change the CURRENT values in the DataTable
    DataRow row = dtCustomers.Rows.Find("ALFKI");
    row["City"] = "Somewhere";
    // ORIGINAL city == Berlin
    // CURRENT  city == New York
    DisplayDataRowVersions(row, "Immediately after I change the City" +
        " from Berlin to New York", "City");

    // Load another DataTable with customer data.
    DataTable dtCustomers2 = new DataTable("Customers");
    adpt.Fill(dtCustomers2);
    DataTableReader dtRdrCustomers = dtCustomers2.CreateDataReader();

    LoadOption opt = LoadOption.PreserveChanges;
    switch (opt)
    {
        case LoadOption.OverwriteChanges:
            // Overwrite ORIGINAL and Overwrite CURRENT values
            dtCustomers.Load(dtRdrCustomers, LoadOption.OverwriteChanges);
            // ORIGINAL city == Miami
            // CURRENT  city == Miami
            row = dtCustomers.Rows.Find("ALFKI");
            ShowVersions (row,
                "Immediately after LoadOptions.OverwriteChanges",
                "City");
            break;
        case LoadOption.Upsert:
            // Keep ORIGINAL and Overwrite CURRENT values
            dtCustomers.Load(dtRdrCustomers, LoadOption.Upsert);
            // ORIGINAL city == Berlin
            /// CURRENT  city == Miami
            row = dtCustomers.Rows.Find("ALFKI");
            ShowVersions (row,
                "Immediately after LoadOptions.Upsert", "City");
            break;
        case LoadOption.PreserveChanges:
            // Overwrite    ORIGINAL and    Keep        CURRENT values
            dtCustomers.Load(dtRdrCustomers, LoadOption.PreserveChanges);
            // ORIGINAL city == Miami
            // CURRENT  city == New York
            row = dtCustomers.Rows.Find("ALFKI");
            ShowVersions (row,
                "Immediately after LoadOptions.PreserveChanges", "City");
            break;
    }
}

To get a first-hand feel for how these settings can be used, try stepping through this code in the debugger. You can try the different LoadOption enumerators by changing the opt variable that appears immediately before the switch-case block. The code shown in Figure 4 invokes the ShowVersions method which simply displays to the output window the original version and current version of a given column.

Changing RowState

A row's state is the major factor that helps determine which rows to update, insert, or delete when the DataAdapter's Update method is invoked. The RowState is also examined by the GetChanges method, as well, to determine which rows to grab. When you make changes to values in a DataSet, ADO.NET handles setting the RowState for you, indirectly setting it to one of the RowState values, such as Modified, Added, or Unchanged.

Sometimes it would be really helpful to be able to set a row's state directly. This is where the DataRow's new SetAdded and SetModified methods make things a little easier. For example, let's assume that you are faced with the situation in which you need to copy a handful of rows from one database to another using ADO.NET. Using ADO.NET 2.0 you could fill a DataTable from a database using the DataAdapter's Fill method, change the rows' RowState settings to Added, and send them down to the second database (assuming it uses the same schema) to be added using a second DataAdapter. In this type of situation you could change the RowState of the rows from Unchanged to Added by invoking each row's SetAdded method.

To demonstrate how these methods work, I have included another example, shown in the code in Figure 5. This code retrieves a rowset of customers and sets two of the row's RowState settings to Modified and another row's setting to Added. Then, I use the DataTable's GetChanges method to create a DataTable containing the rows with a RowState of Modified and store the number of rows. I then get the number of rows that were added and display them using a MessageBox.

Figure 5 Changing a Row's State (Close Figure 5)
DataTable dtCustomers = new DataTable("Customers");

using (SqlConnection cn = new SqlConnection(cnStr))
{
    SqlCommand cmd = new SqlCommand(sqlAllCustomers, cn);
    SqlDataAdapter adpt = new SqlDataAdapter(cmd);
    adpt.Fill(dtCustomers);
    dtCustomers.PrimaryKey = new DataColumn[] {
        dtCustomers.Columns["CustomerID"] };
}

// Change the RowState of a few rows
DataRow row = dtCustomers.Rows.Find("ALFKI");
row.SetModified();
row = dtCustomers.Rows.Find("BOLID");
row.SetModified();
row = dtCustomers.Rows.Find("ANTON");
row.SetAdded();

int modRows = dtCustomers.GetChanges(
    DataRowState.Modified).Rows.Count;
int addRows = dtCustomers.GetChanges(DataRowState.Added).Rows.Count;

StringBuilder bldr = new StringBuilder();
bldr.Append(modRows.ToString());
bldr.Append(" row(s) were modified.");
bldr.Append(Environment.NewLine);
bldr.Append(addRows.ToString());
bldr.Append(" row(s) were added");
MessageBox.Show(bldr.ToString());

The SetAdded and SetModified methods of the DataRow only work on rows that are unchanged. Another situation in which these come in handy is when you receive a DataSet or a DataTable from a Web service and the rows are all marked as Unchanged. If you intend to make additions or updates to a database based on the DataTable, you could set the RowState using these new methods. Otherwise, if you leave the RowState as Unchanged, the DataAdpater's Udpate method would not send the rows to either the UpdateCommand or the InsertCommand.

0 to 60

One of the best new features of ADO.NET 2.0 is neither a new method nor a new class but rather a focused performance improvement. One of the big knocks on the DataSet and the DataTable has been how slow they can be to load, especially when the number of rows gets large (100, 1,000, 10,000, or more). It can be just as bad, if not worse, when trying to traverse a large DataTable. To address this practical limitation, a lot of effort went into the creation of a faster indexing engine in ADO.NET 2.0. The rewrite of the indexing engine in ADO.NET has resulted in a huge performance boost in all areas including loading and merging DataTables. Figure 6 shows some sample code that can be run in either Visual Studio® .NET 2003 (using ADO.NET 1.1) or Visual Studio 2005 (using ADO.NET 2.0). I ran a benchmark comparison of this code in both environments with varying numbers of rows.

Figure 6 Speed Test (Close Figure 6)
DataTable dt = new DataTable("foo");
DataColumn pkCol = new DataColumn("ID", Type.GetType("System.Int32"));
pkCol.AutoIncrement = true;
pkCol.AutoIncrementSeed = 1;
pkCol.AutoIncrementStep = 1;
dt.Columns.Add(pkCol);
dt.PrimaryKey = new DataColumn[] { pkCol };
dt.Columns.Add("SomeNumber", Type.GetType("System.Int32"));
dt.Columns["SomeNumber"].Unique = true;

int limit = 1000000;
int someNumber = limit;
DateTime startTime = DateTime.Now;
for (int i = 1; i <= limit; i++)
{
    DataRow row = dt.NewRow();
    row["SomeNumber"] = someNumber—;
    dt.Rows.Add(row);
}
TimeSpan elapsedTime = DateTime.Now - startTime;
MessageBox.Show(dt.Rows.Count.ToString() +
    " rows loaded in " + elapsedTime.TotalSeconds + " seconds.");

The code in Figure 6 creates a DataTable, adds two columns to it, and then loops 1,000,000 times adding a row to the DataTable in each iteration. Once the loop is completed, the elapsed number of seconds is displayed to the user using the MessageBox method. I ran this test for a few different iterations in both ADO.NET 1.1 and ADO.NET 2.0. My results are shown in Figure 7.

Figure 7 Speed Test Results (Close Figure 7)

Iterations | ADO.NET 1.1 | ADO.NET 2.0
10,000 | 0.20 | 0.20
100,000 | 7.91 | 3.89
1,000,000 | 1831.01 | 23.78

The speed is even better if there are no constraints on the DataTable. For example, when I removed the Unique constraint, I was able to load a million rows in just over a second in both versions of the environment. It is also important to note that I only loaded two columns and the value for the SomeNumber column was a decrementing but still sequential integer. While your results will vary from mine, the key point to take away is that the indexing engine for ADO.NET 2.0 performs significantly faster— so much faster that it is now completely realistic to at least consider using a DataTable to contain a million rows.

Wrapping It Up

In ADO.NET 2.0, there's better performance in areas that were lagging in the previous version, such as loading large numbers of rows. Several new features have been added to make development easier. The DataTable class has gained several methods that already existed on the DataSet class and there is even a new DataTableReader class. In the next installment of the Data Points column, I will continue the discussion of ADO.NET 2.0 by examining how binary serialization improves performance, how to take advantage of batch updates, the new DataView features, the new SqlConnectionBuilder class, and much more.

Send your questions and comments for John to mmdata@microsoft.com.

John Papa is a Senior .NET Consultant with ASPSOFT and a baseball fanatic who spends most of his summer nights rooting for the Yankees with his family and his faithful dog, Kadi. He has authored several books on ADO, XML, and SQL Server, and can often be found speaking at industry conferences such as VSLive or blogging at codebetter.com/blogs/john.papa.


Basic Instincts: Programming I/O with Streams in Visual Basic .NET

Basic Instincts
Programming I/O with Streams in Visual Basic .NET
Ted Pattison

Contents
What is a Stream?
Programming with Streams
Text-Based I/O Using a Reader and Writer
Stream-Based Programming with XML
Using Streams with Network-Based Programming
Wrap-Up

If you are just now migrating from Visual Basic® 6.0 to Visual Basic .NET, be prepared for a change, because the way you program file I/O is very different. The good news is that an idiosyncratic, proprietary approach has been replaced by an elegant and reusable one that has become an industry standard.

I/O programming changes in both Visual Basic .NET and in the Microsoft® .NET Framework represent far more than just new syntax and method names. There's a complete paradigm shift because everything is based on the concept of a stream.

In this column, I will introduce stream-based programming and the .NET Framework classes associated with it. Along the way, I will walk through several practical scenarios. You'll see that streams create a consistent I/O programming style regardless of the source or destination of your data.

What is a Stream?

A stream is an object that represents a generic sequence of bytes. Any type of data can be stored as a sequence of bytes, so the details of writing and reading data can be abstracted. For example, you can write a generic piece of code that writes an XML document to a stream. Then you can use that code in an application to write the XML document to several different types of streams, such as an in-memory buffer, a file, or even a network connection.

While a stream object works at the byte level, it's fortunate that programmers aren't typically required to write and read data on a byte-by-byte basis. Imagine how hard it would be to take the data associated with a high-level document, such as an invoice, and transform it into a byte array. While that sounds like a challenging task, it would be even more difficult to write the code to reassemble the invoice from the byte array when it's time to present this data to the user.

The .NET Framework Class Library includes several complementary pairs of reader and writer objects that improve the usefulness and ease of use of streams. Figure 1 illustrates the typical relationship between writer objects, stream objects, and reader objects.

Figure 1 Reader and Writer Objects
Figure 1 Reader and Writer Objects

To write data into a stream in an efficient manner, create a writer object and initialize it so it is associated with a target stream object. A writer object exposes high-level methods (such as WriteLine) that allow you to think in terms of high-level data structures (such as strings) instead of concerning yourself with the nitty-gritty details of actually writing bytes. Behind the scenes, the writer object does the work of crunching the data into a byte array and writing it into the target stream.

When it's time to retrieve the data, you simply call high-level methods on the reader object (for example, ReadLine). The reader object responds by fetching bytes from the stream and reassembling the data back into its original form.

Before looking at any code, let's discuss where stream objects come from. The .NET Framework Class Library contains a Stream class in the System.IO namespace. The Stream class itself is an abstract class (a MustInherit class), meaning it cannot be directly instantiated to create an object. Its sole purpose is to serve as a base class for other Stream-derived classes. Actual stream objects must be instantiated from creatable classes that inherit either directly or indirectly from the Stream class.

A key point is that the Stream class has been designed to provide a common implementation and a standard programming contract for every stream object. The .NET Framework provides several concrete implementations of the Stream class, a subset of which are shown in Figure 2. As you can see, each of these Stream classes inherits from the Stream class and extends it to provide a unique implementation of a stream.

Figure 2 Inheriting from the Stream Class
Figure 2 Inheriting from the Stream Class

For example, you can use the MemoryStream class to create stream objects to store and retrieve data from an in-memory buffer. You can use the FileStream class to create stream objects to read or write data in a file on the local file system. You can also use stream objects created from one of the network stream classes to read or write the body of a network message.

Programming with Streams

Let's start with an example of programming against a stream in the raw without using a reader or a writer. While this style of programming doesn't yield high levels of productivity, this simple example will begin to build your understanding of stream-based programming from the ground up.

Examine the code in Figure 3. It creates and initializes a MemoryStream object with a capacity of four bytes. The code then writes four bytes of data into the MemoryStream, one byte at a time. You can imagine how programming at the byte level would become increasingly tedious and inefficient as the complexity of the data structure increases.

Figure 3 Memory Stream Object (Close Figure 3)
'*** create memory stream object
Dim stream1 As Stream = New MemoryStream(4)

'*** write data to stream
stream1.WriteByte(2)
stream1.WriteByte(4)
stream1.WriteByte(6)
stream1.WriteByte(8)

'*** clear stream buffer (if it has one)
stream1.Flush()

'*** read data from stream
stream1.Position = 0
Dim result As Integer
result = stream1.ReadByte()
Do While result <> -1
  Console.WriteLine(result)
  result = stream1.ReadByte()
Loop

'*** close stream as soon as work is complete
stream1.Close()

Next, note the call to the Flush method of the Stream class. This is a method used with buffered streams (not all streams are buffered) to proactively push any data from the buffer to its destination or to clear the buffer.

Now, examine the code that reads data from the stream. You should notice how the code sets the Position property back to 0 before moving through the array of bytes. As you can see, some stream classes, such as MemoryStream, support cursor-like navigation where your code can move through the byte array, one byte at a time, and extract the data using the ReadByte method.

Finally, notice that the Close method is called as soon as the application is finished using the stream object. Keep in mind that many types of stream objects open expensive resources. You should get in the habit of closing a stream as soon as you are finished using it. For example, the call to the Close method on a FileStream object is what releases the OS-level lock on the file and allows the file to be accessed by other stream objects and by other applications.

Text-Based I/O Using a Reader and Writer

Now that you are familiar with programming a stream in the raw, it's time to introduce a writer object and a reader object to make things easier when you are programming text-based I/O. In particular, you can write text to a stream and read it back using a pair of classes that exist in the System.IO namespace named StreamWriter and StreamReader.

The StreamWriter and StreamReader classes inherit from two other abstract base classes: TextWriter and TextReader. The class hierarchy has been designed like this so you can write generic I/O code using TextWriter and TextReader that can write and read text to and from either a stream object or a StringBuilder object. If you wanted to work with text inside a StringBuilder object, you would then use the StringWriter class and the StringReader class instead of the StreamWriter class and StreamReader class, respectively.

The code in Figure 4 shows an example of a generic method named WriteSpecialMessage that accepts a single parameter of type TextWriter. This code demonstrates the polymorphic abilities of the TextWriter base class, because the WriteSpecialMessage method can be used to write text to either a stream object or a StringBuilder instance.

Figure 4 WriteSpecialMessage (Close Figure 4)
Imports System
Imports System.IO
Imports System.Text

Class MyApp

  '*** can be used to write to Stream or StringBuilder objects
  Shared Sub WriteSpecialMessage(ByVal writer As TextWriter)
    writer.WriteLine("Hello")
    writer.WriteLine("Goodbye")
  End Sub

  Shared Sub Main()

    '*** write text message to Stream
    Dim stream1 As New MemoryStream()
    Dim writer1 As New StreamWriter(stream1)
    WriteSpecialMessage(writer1)
    writer1.Close

    '*** write text message to StringBuilder
    Dim builder1 As New StringBuilder()
    Dim writer2 As New StringWriter(builder1)
    WriteSpecialMessage(writer2)
    writer2.Close

  End Sub

End Class

Now let's move ahead to a typical scenario in which you need to create a new text file on the local file system. You can start by using the FileStream class, which provides support for general file I/O. The following code shows an example of how to create and initialize a FileStream object using a constructor that accepts three parameters—a file path, a file mode, and a file access setting:

Dim fs As New FileStream("MyData.txt", _
                         FileMode.Create, _
                         FileAccess.Write)
This code creates a new file in the current directory and opens it up for write access. Note that the System.IO namespace contains two utility classes, named File and Path, that make it easier to open files, find and search through directories, generate file paths, and determine whether a file already exists.

Once you have opened the file by creating and initializing a FileStream object, you can create a new StreamWriter object to write text into the file. When you create a StreamWriter object, you associate it with a stream by passing a reference to the target stream object in the StreamWriter constructor. Figure 5 shows how to use a StreamWriter object together with a FileStream object. Note that the call to the Flush method on a StreamWriter object is required to push the data into the destination stream and a call to the Close method on the FileStream object closes the file and releases the OS-level lock.

Figure 5 StreamWriter Object and FileStream Object (Close Figure 5)
Dim fs As New FileStream("MyData.txt", FileMode.Create, FileAccess.Write)

'*** now create and initialize a writer object
Dim writer As New StreamWriter(fs)

'*** now use the writer object
writer.WriteLine("Hello")

'*** call Flush on writer when writing is done
writer.Flush()

'*** call Close on stream to save and close file
fs.Close()

Reading text is very similar and also fairly easy. You create another FileStream object, except this time, you initialize it for read-only access. Instead of creating a StreamWriter object, you create a complementary StreamReader object. Figure 6 uses a StreamReader object to read text from the file line by line.

Figure 6 Reading Line by Line with StreamReader (Close Figure 6)
Dim fs As New FileStream("MyData.txt", FileMode.Open, FileAccess.Read)

Dim reader As New StreamReader(fs)

'*** read text out of file one line at a time
Do Until reader.Peek() = -1
  Console.WriteLine(reader.ReadLine)
Loop

'*** make sure to close stream when work is complete
fs.Close()

If you don't want to read text from files on a line-by-line basis, you can retrieve the entire contents of a text file as a single string value using a StreamReader object's ReadToEnd method.

As mentioned previously, the File class provides some useful shared methods to simplify access to files. Two of these methods, OpenRead and OpenWrite, make it easy to open files as I've been doing, but with less code. For example, instead of using the following code to open MyData.txt for reading:

New FileStream("MyData.txt", FileMode.Open, FileAccess.Read)
I can use the equivalent:
File.OpenRead("MyData.txt")

Stream-Based Programming with XML

Now let's perform I/O with something more complex: an XML document. As you will see, the programming techniques revolve around a similar set of objects. Imagine you are required to read an XML document that conforms to the following structure:

<?xml version="1.0" encoding="utf-8" ?>
<Dancers>
  <Dancer>Fritz</Dancer>
  <Dancer>Keith</Dancer>
  <Dancer>Ted</Dancer>
  <Dancer>Aaron</Dancer>
</Dancers>
The fastest way to read this data from its XML format is to use a stream-based technique based on the XmlTextReader class defined in the System.Xml namespace. When you create a new XmlTextReader object, you can explicitly initialize it with a FileStream object and a StreamReader object:
'*** create and initialize stream object
Dim stream As New FileStream("MyData.xml", _
                             FileMode.Open, _
                             FileAccess.Read)

'*** create and initialize StreamReader using stream
Dim reader1 As New StreamReader(stream)

'*** create and initialize XmlTextReader using StreamReader
Dim reader2 As New XmlTextReader(reader1)

Of course, there is an easier way to accomplish the same goal. You can create and initialize an XmlTextReader object using a string pointing to the target file path. When you do this, the XmlTextReader object automatically creates and manages a FileStream object and a StreamReader object behind the scenes:

'*** this code does exactly what the previous code does.
Dim reader As New XmlTextReader("MyData.xml")

Once you have initialized an XmlTextReader object with a specific XML file, you can begin to access the data inside using a stream-based approach. You can move forward node by node to read the contents of the XML document, but you cannot move backwards. The code that is shown in Figure 7 reads data from an XML file using this particular approach.

Figure 7 Using XmlTextReader (Close Figure 7)
Dim reader As New XmlTextReader("MyData.xml")

'*** determine how XmlTextReader deals with whitespace.
reader.WhitespaceHandling = WhitespaceHandling.None
'*** move through XML content node by node
reader.MoveToContent()

Do While reader.Read()
  Select Case reader.NodeType
    Case XmlNodeType.Element
      Console.Write(reader.Name & ": ")
    Case XmlNodeType.Text
      Console.WriteLine(reader.Value)
  End Select
Loop

'*** close reader when done to release resources
reader.Close()

Using Streams with Network-Based Programming

So far you have seen how streams allow you to read and write into memory buffers as well as files. Now I would like to move on to other scenarios in which you are required to write and read the body of network messages. My goal here is to demonstrate how stream-based programming for the network is consistent with what you have already seen in this column.

Imagine you are writing a .NET-based client app that needs to retrieve data from a Web server using HTTP. For client-side HTTP programming, the .NET Framework provides classes in the System.Net namespace that let you submit HTTP requests to a Web server using operations such as GET and POST.

Using this approach, you can use a WebRequest object to transmit GET and POST operations to a Web server. Once you have initialized the WebRequest object, you call a method named GetResponse to send the HTTP request across the network to the Web server. The GetResponse method doesn't return until a response is received from the Web server. When the GetResponse method does complete, it returns a reference to a WebResponse object, which allows the client application to retrieve data sent by the Web server by providing stream-based access to the body of the HTTP response message.

Now take a look at the code in Figure 8, which sends an HTTP request using a GET operation and then retrieves text from the body of the HTTP response message returned from the Web server. As you can see, you create a WebRequest object using a shared factory method named Create. Once the object has been created, you can determine the type of HTTP operation by assigning a string value, such as GET or POST, to the Method property. You will also need to add security code at this point for authenticating the client application, unless you can make the assumption that all requests to the Web server can be made using anonymous access.

Figure 8 Using a GET (Close Figure 8)
Dim req As WebRequest
req = WebRequest.Create("http://LocalHost/TestApp/CustomerList.aspx")
req.Method = "GET"

'*** submit synchronous HTTP request to Web server
Dim rsp As WebResponse = req.GetResponse

'*** WebResponse provides stream-based access to body
Dim str As Stream = rsp.GetResponseStream
Dim reader As New StreamReader(str)
Console.WriteLine(reader.ReadToEnd())
reader.Close()
rsp.Close()

When you call GetResponse, the WebRequest object responds by submitting an HTTP request message to the Web server. GetResponse is a synchronous method that blocks for the duration of the trip to the Web server and back. When the HTTP response message is returned from the Web server, the GetResponse method completes and returns a WebResponse object to its caller.

When you call GetResponseStream, the WebResponse object returns a reference to a stream object that allows you to access the body of the incoming HTTP response message. This, in turn, lets you use stream-based techniques, such as the ones that use a reader object to work with text and XML. A high-level view of how all these pieces fit together is shown in Figure 9.

Figure 9 Stream-Based Access to the HTTP Response
Figure 9 Stream-Based Access to the HTTP Response

Before I conclude, I'd like to shift focus to the server-side code written to handle an incoming HTTP request in an ASP.NET page. While the classes you will use in this scenario have been written by the ASP.NET team, they offer a consistent programming approach because they too are based on streams.

When an ASP.NET page is processed in response to an incoming HTTP request, the body of the HTTP request message is exposed through the InputStream property of the ASP.NET HttpRequest object. Likewise, you can write into the body of the HTTP response message that will be sent back to the client by using the OutputStream property of the ASP.NET HttpResponse object. Figure 10 is an ASP.NET page that uses a stream-based technique involving the OutputStream property and a StreamWriter object to send a few lines of text back to the client.

Figure 10 Using OutputStream and StreamWriter (Close Figure 10)
<% @Page Language="vb" %>
<% @Import namespace="System.IO" %>

<script runat=server>

  Sub Page_Load(ByVal sender As Object, ByVal args As EventArgs)
    WriteCustomerList(Response.OutputStream)
  End Sub

  Sub WriteCustomerList (ByVal str As Stream)
    Dim writer As StreamWriter = New StreamWriter(str)
    writer.WriteLine("Nancy Davolio")
    writer.WriteLine("Andrew Fuller")
    writer.WriteLine("Ted Pattison")
    writer.Flush()
  End Sub

</script>

As you can see, the generic method WriteCustomerList can be used to write text into any type of stream. You can take this example even further and develop ASP.NET pages and custom HttpHandler objects that use the OutputStream property to write other types of data, such as XML documents and binary data, into the body of the HTTP response message.

Wrap-Up

This month I took an introductory look into programming I/O in the .NET Framework with Visual Basic .NET. As you have seen, all I/O work is built on top of the concept of a stream. Streams are valuable because they abstract away the details of where your data is going to and coming from. You can code generic routines that can be used to write and read binary data, text, and XML data to and from memory buffers, files, and network messages.

You have also seen how you can complement streams by using a pair of writer and reader objects, which simplify stream-based I/O when you are working with more complicated kinds of data structures. You should now be ready to augment your I/O programming repertoire with streams.

Send your questions and comments for Ted to instinct@microsoft.com.

Ted Pattison is an author and trainer who delivers hands-on training through PluralSight, provides consulting through Ted Pattison Group, and is the author of several books.


Cutting Edge: A Quick Tour of Themes in ASP.NET 2.0

Cutting Edge
A Quick Tour of Themes in ASP.NET 2.0
Dino Esposito

Code download available at: CuttingEdge0511.exe (116KB)
Contents
Page Themes
Theme Internals
Skinning Controls
Special Themes Features
Summing It Up

It's much easier to build a rich user interface into your Web application in ASP.NET 2.0 than it was in previous versions. Master Pages let you build pages based on existing templates of markup and code. ASP.NET 2.0 wizards make implementing navigation easier. In addition, you can add personalization to ASP.NET 2.0 pages thanks to features like the user profile that I covered last month. This month I'll take a quick tour through another feature that makes implementing ASP.NET applications easier—themes—which let you skin control pages and achieve visual consistency with minimal effort. Note that I'm working with a beta release of ASP.NET 2.0 and some details discussed here may change by the final release.

Page Themes

The beauty of ASP.NET 2.0 themes is that, much like Windows® XP themes, they allow you to radically change the appearance of a set of pages by applying minimal changes. In a themed control, visual properties can be set and changed in a single shot. With ASP.NET themes, you can build skinned controls as long as the control supports templates. I'll return to this in a moment.

A theme is an attribute of the page class that you can set in various ways. Once the theme is set, the page ensures that all of its controls will render according to the visual settings defined in the theme. Themes let you keep style markup out of your ASPX code so you can write pages without concerning yourself with the style of controls and the graphics around them. Later, designers can create the theme.

In ASP.NET 2.0, a theme is made up of various files all installed in a given folder. The name of the folder determines the name of the theme. In the folder, there are a few types of files: a skin file, a CSS file, and optionally a subfolder of images or other auxiliary files such as XSLT and text files. Figure 1 shows an example. Note that the theme displayed—SmokeAndGlass—was originally included in Beta 1 of ASP.NET 2.0, but was later removed. I just made a backup copy of the files and kept them as a reference.

Figure 1 Custom Themes in ASP.NET 2.0
Figure 1 Custom Themes in ASP.NET 2.0

To be recognized by an application, a theme folder must either be located under the App_Themes folder below the application's root, or in a global folder. A theme located under the App_Themes folder at the root of the application is said to hold local themes as only the specific application can use it. Global themes are contained in child directories located under the following path:

%WINDOWS%\Microsoft.NET\Framework\[version]\
   ASP.NETClientFiles\Themes
The [version] placeholder indicates the version number of the ASP.NET 2.0 build you have installed. For Beta 2, it is v2.0.50215. Expect it to be different if you have installed any post-Beta 2 releases.

Theme Internals

A skin is a markup file made of the control definitions like those you find in an .aspx file. One way to create a skin file is to simply create a temporary page, dropping controls in and styling as appropriate. When you're finished, just copy and paste the markup source of the control file to a skin file (see Figure 2), making sure to remove each control's ID property from the markup.

Figure 2 Create a Skin (Close Figure 2)
<asp:GridView runat="server" SkinID="Classic"
        CellPadding="4" ForeColor="#333333" GridLines="None">
    <FooterStyle BackColor="#507CD1" Font-Bold="True" ForeColor="White" />
    <RowStyle BackColor="#EFF3FB" />
    <PagerStyle BackColor="#2461BF" ForeColor="White"
        HorizontalAlign="Center" />
    <SelectedRowStyle BackColor="#D1DDF1" Font-Bold="True"
        ForeColor="#333333" />
    <HeaderStyle BackColor="#507CD1" Font-Bold="True" ForeColor="White" />
    <EditRowStyle BackColor="#2461BF" />
    <AlternatingRowStyle BackColor="White" />
</asp:GridView>

A skin contains standard definitions for as many controls as you want. You can also define multiple skins per control. In this case, you add an extra attribute to the control definition—SkinID. The SkinID attribute names the control skin and allows controls to select that template at run time. Consider the following GridView fragment in an .aspx page:

<asp:GridView runat="server" id="grid" skinID="Classic" />
The SkinID attribute selects the specified skin from the bound theme, if there is one. Should the specified skin ID not match an existing skin for the control, you will get a warning from Visual Studio® 2005, and the skin ID will be ignored, thus resulting in the control being displayed without a theme.

A theme can exist without a CSS file, and vice-versa, of course. Skin files are not a superset of CSS per se, but rather of control property sets, and like regular rendering they result in inline CSS on markup elements.

Sound coding practices dictate that skin files should not contain behavioral properties such as AllowPaging. However, this is simply a recommendation; if you set AllowPaging in the skin file, it will work just fine, but storing behavioral properties in the skin like this violates the logical separation between style and behavior. You should also note that it is the control that dictates which of its properties can be themed.

Skinning Controls

You typically bind a theme to a page using the Theme attribute on the @Page directive:

<% @Page ... Theme="SmokeAndGlass" %>
You can also declaratively bind a theme to all the pages in a given folder by moving the setting to the folder-specific Web.config file:
<pages theme="SmokeAndGlass" />

A given theme can be bound to a page in two different ways: as a stylesheet theme or as a customization theme. Stylesheet themes are designer-supported and mimic CSS in a way that allows the developer to abstract styles to one location (for specific controls, you can override the style on the control directly). Aside from this and from the keyword you use, the difference is all in the timing. If you use the Theme attribute, you are binding a customization theme so the ASP.NET runtime will apply the settings of the theme only after processing the declarative attributes of controls. A stylesheet theme is bound by using the Stylesheet attribute, like so:

<% @Page ... StylesheetTheme="SmokeAndGlass" %>
In this case, the theme is applied before processing the declarative attributes. To better understand the difference, let's take a look at the following markup:
<!-- In the skin file -->
<asp:DataGrid runat="server" backcolor="red" />

<!-- In the page -->
<asp:DataGrid runat="server" backcolor="yellow" />
If the control belongs to a page bound to a customization theme, the grid will be displayed with the background color red as set in the theme. If the page is styled with a stylesheet theme, yellow will be the color of the grid's background, regardless of the color set in the theme. Note that any properties set through a theme can also be overridden programmatically by handling the Page_Load event if you want.

The GridView control supports the EmptyDataTemplate property for you to specify the markup that should be used when the control is bound to an empty data source:

<asp:GridView ...>
  ...
  <EmptyDataTemplate>
     <table style="border:solid 1px black"><tr><td>
       <img runat="server" align="absmiddle" src="images/critical.gif">
       <b>Hello!</b><br>I'm a <b>GridView</b> control.
       Unfortunately, though, at this time I have no data to display.
     </td></tr></table>
  </EmptyDataTemplate>
</asp:GridView>

The image critical.gif will be loaded from the Images folder below the theme folder (as you saw in Figure 1). Figure 3 shows the same grid control skinned differently. The figure shows the grid bound to distinct skins in the same theme, but you can easily move the two skins to distinct theme files.

Figure 3 Empty GridView Control Skinned Two Ways
Figure 3 Empty GridView Control Skinned Two Ways

Special Themes Features

Loading themes programmatically when needed for individual pages couldn't be easier. All you have to do is set the Theme property on the Page class to the string that represents the name of the theme (stylesheet themes cannot be set in this manner, for they require an override of the StyleSheetTheme property on the Page class). The only trick is that you must set the theme in the PreInit event of the page. PreInit is a new event introduced in ASP.NET 2.0 which is fired just before Init, typically the first point in the control's lifecycle, and therefore a point where property sets for the theme can be applied before a control starts to manipulate properties. At this time PreInit is called, the page's control tree is already built, but the individual controls' trees themselves may not be.

By hooking the PreInit event, you can set the theme to a predetermined theme name or to a theme name that you get from the outside world—the user's profile, a dropdown list, the query string, or any number of other places. The following illustrates a good code pattern for loading themes dynamically into a page:

Sub Page_PreInit(ByVal sender As Object, ByVal e As EventArgs)
    Dim theme As String = GetThemeToApply();
    If theme = "" Then Return
    Me.Theme = theme
End Sub
A theme can also be used to style custom controls, as long as custom controls are properly registered in the skin file through the @Register directive or in the configuration file.

A control will be affected by a relevant theme unless its EnableTheming property is set to false (the default is true). When designing a control, you can mark one or more properties as theme-resistant by applying the Themeable attribute and setting it to false:

Public Class MyControl
        Inherits WebControl
    <Themeable(false)>
    Public Property TextFont As Font
        ...
    End Property
End Class

This normally happens if it is essential for the user interface of the control that the property be set programmatically or declaratively but always under the control of the developer or page code. Of course, properties that are not stylistic or that are security conscious (for example, DataSourceID) should not be themeable. You would not want to allow a third party to change a control's DataSourceID through a theme.

Summing It Up

There is a lot more to themes than I've covered here, such as programmatically setting skinIDs and how themes and stylesheet themes interact with dynamic controls, but I hope this quick tour has helped to instill an appreciation for how useful this new feature can be. Personalization is important for users, but it also helps application designers to produce a more flexible and configurable framework. Skins and themes are ASP.NET 2.0 facilities at your disposal that really make personalization and customization much easier to accomplish.

Send your questions and comments for Dino to cutting@microsoft.com.

Dino Esposito is a mentor at Solid Quality Learning and the author of Programming Microsoft ASP.NET 2.0 (Microsoft Press, 2005). Based in Italy, Dino is a frequent speaker at industry events worldwide. Get in touch with Dino at cutting@microsoft.com or join the blog at weblogs.asp.net/despos.


Service Station: An XML Guru's Guide to BizTalk Server 2004, Part I

Service Station
An XML Guru's Guide to BizTalk Server 2004, Part I
Aaron Skonnard

Contents
BizTalk State of Affairs
Theory vs. Reality
BizTalk Server Architecture
A Service-Oriented Foundation
Publish-Subscribe Engine
Contract-Driven Messaging
Port Extensibility
Wrap-Up

Who would have believed that XML, such a seemingly trivial technology, could revolutionize an industry? It may have seemed like a long shot in the beginning, but the XML aficionados saw something special and pragmatic right away—a sort of duct tape for the world's information systems. But not all developers felt this way. Some were disappointed by the lack of tool support that would bring XML to life.

Given its place today, however, I don't think anyone could say that XML was all hype. It has revolutionized the way we think about distributed computing, given birth to Web services and service orientation, and continually spawns newer, better tools. One such tool, BizTalk® Server 2004, is poised to finally bring some of XML's benefits to the masses by providing some long-awaited magic.

BizTalk State of Affairs

BizTalk Server 2000 laid the foundation for simplifying common XML use cases in the enterprise. But, needless to say, the final package had some warts. It was hard to install, configure, and operate. Its available tools confused developers and, as a result, was never able to gain significant traction.

Since that humble beginning, the product has evolved significantly. BizTalk Server 2002 came shortly after the initial release, and provided some minor improvements. The big change, however, came with BizTalk Server 2004 when the entire product was rewritten for the Microsoft® .NET Framework. BizTalk Server 2004 came with some major simplifications and a new toolset integrated with Visual Studio® .NET—just the right ingredients for a good developer experience.

And future versions look even better. Microsoft recently released a preview of the upcoming BizTalk Server 2006 (code-named "Pathfinder"), which continues to build on the foundation of .NET, while introducing additional improvements and simplifications. One of the primary goals of the 2006 release is to simplify the user experience, especially the installation, configuration, and operation processes, which have caused so much pain in the past. I've heard some say one of their primary goals was to achieve a SQL Server™-like Next?Next?Next installation. The current release looks very promising.

Theory vs. Reality

One of the first things I noticed after playing with BizTalk Server 2004 is how grounded the product is in reality. Most businesses don't have the luxury of being able to throw away existing investments, so integrating legacy systems with a wide range of communication protocols and data formats is a very big challenge.

While advocates of Web services believe that their approach is the solution to these enterprise challenges, that approach alone would require updating all legacy systems (at least those that don't speak Web services today). If Web services were the answer, everything would have to speak SOAP and the Web services family of specifications. That's a tall order for most enterprises. In addition, most Web services stacks only support a few protocols and are not message-oriented by design, which can limit you when dealing with certain integration scenarios.

The BizTalk folks took a different approach to this problem. Instead of enforcing common protocols, BizTalk Server supports a myriad of protocols through its adapter framework. BizTalk Server provides numerous adapters in the box that address some of the most mainstream use cases, but what's really exciting is the suite of nearly 100 partner adapters you can plug into. You should be able to find one for just about any common enterprise application or technology in use today. Of course, in addition to all of this flexibility, BizTalk Server doesn't require the use of SOAP or WS-*, or even XML. Through BizTalk pipelines and maps, you can support a variety of different data formats to fit your needs, including common flat files and Electronic Data Interchange (EDI).

BizTalk also embraces core XML technologies, such as XPath and XSLT, to deal with tough problems like versioning or alternate data formats to fit a particular customer's needs. In addition, BizTalk provides sophisticated integration with today's evolving Web services technologies when dealing with newer SOAP-savvy applications. Overall, BizTalk Server 2004 offers a message-oriented approach grounded in today's realities.

BizTalk Server Architecture

The BizTalk Server 2004 architecture is neatly organized into two primary buckets: messaging and orchestration. The messaging layer is focused on the details of sending and receiving messages or in other words, "connecting the dots" between applications. In order for an application to participate with BizTalk Server, it needs to be able to communicate via messages. The orchestration layer is focused on coordinating numerous individual message exchanges (see Figure 1), dealing with sequencing and flow, error handling and transactions, and so on. Most BizTalk Server 2004 features and technologies fit nicely into one of these buckets.

Figure 1 Messaging vs. Orchestration
Figure 1 Messaging vs. Orchestration

The messaging architecture provides the foundation for the entire product. You can tap directly into the messaging layer (more on this shortly), or you can interface with it indirectly using the orchestration layer. Orchestrations build on the messaging layer to send and receive the messages used by the business process. Orchestrations are a powerful way to automate complex business processes, but the connectivity and integration benefits come from the underlying messaging components. Nonetheless, there are use cases for BizTalk Server where orchestrations aren't needed, such as in many simple one-way application connectivity scenarios.

It should be no surprise that the messaging architecture is built on an XML foundation. BizTalk Server 2004 can deal with a variety of different message types, but XML is considered the primary or default message type. As a result, BizTalk Server 2004 leverages XML Schema as the primary type system for defining message schemas. And BizTalk Server 2004 provides excellent tool support, integrated with Visual Studio .NET, for working with these XML technologies. XML savvy developers will feel right at home working on this foundation.

A Service-Oriented Foundation

The other thing that really struck me after spending more time with BizTalk Server 2004 was how service-oriented it really is. Most of the folks who promote the tenets of service orientation do so in the context of describing Web services applications and designs. But when you apply the tenets of service orientation to the BizTalk Server architecture, you'll notice an even better fit.

With BizTalk Server, messages are sent and received via ports (see Figure 2). What happens in between is a complete black box to the sender and receiver. Messages are received by "receive ports" and sent via "send ports." Typically, orchestrations are associated with receive ports and are executed when messages arrive. The orchestration may send a result message (or multiple messages) to another system through send ports.

Figure 2 BizTalk Ports and Messages
Figure 2 BizTalk Ports and Messages

Once you have BizTalk Server 2004 installed, you can create and configure ports within the BizTalk Explorer in Visual Studio .NET. Figure 3 shows how to configure a receive port location (which reads from a file directory on disk) while Figure 4 shows how to configure a send port (which transmits to an FTP site). Each one configures the transport (adapter) that will be used along with the address where messages will be sent and/or received.

Figure 3 Configuring a Receive Port Location
Figure 3 Configuring a Receive Port Location

Once the ports are created and configured, consumers who want to integrate with the business process must be informed of the proper message types to send to the receive ports and the message types they will receive back from send ports. With BizTalk Server, this information is shared with consumers using XML Schema, Web Services Description Language (WSDL), and Business Process Execution Language (BPEL) definitions.

Figure 4 Configuring a Send Port
Figure 4 Configuring a Send Port

As you can see, BizTalk Server deals directly with explicit boundaries between applications. In many ways, BizTalk Server is the boundary between the various enterprise applications it orchestrates. Orchestrations also have explicit boundaries (for example, you invoke them by sending messages to ports). The participating applications in a BizTalk environment share schema and contract, not implementation details, to negotiate message exchanges. In fact, BizTalk Server itself knows nothing about how the various participating applications were implemented. Participating applications are completely autonomous and black box to BizTalk Server—they could be running on a variety of different platforms and technologies. New enterprise applications can be added to the mix without affecting existing ones since it's simply a matter of configuring a new message exchange. These traits provide you with a model that can adapt to change over time, which is truly the Zen of service orientation.

Publish-Subscribe Engine

One of the key mechanisms behind the BizTalk Server messaging layer is its flexible publish-subscribe engine. At the heart of BizTalk Server is a SQL Server database, referred to as the MessageBox (see Figure 5). All of the messages that pass through BizTalk Server are temporarily stored in the MessageBox until they are finished being processed. The receive ports have the job of persisting incoming messages to the MessageBox while the send ports have the job of retrieving messages from the MessageBox for transmission.

Figure 5 Publish-Subscribe in Action
Figure 5 Publish-Subscribe in Action

The MessageBox is always in the loop, even for simple request/response style operations. Here's a typical scenario: a message enters a receive port and it's saved to the MessageBox. Then BizTalk Server notices that the message arrived and executes the associated orchestration. The orchestration generates a response message, which is then saved back to the MessageBox. Then a send port picks up the new message and transmits it back to the original sender. So the question is how do send ports and orchestrations know whether to respond to messages arriving in the MessageBox?

This is done through message subscriptions. Both send ports and orchestrations define filters to subscribe to messages entering the MessageBox. When a message hits the MessageBox, all known subscriptions are evaluated to identify matches. The message will then be routed to each orchestration or send port with a matching subscription, as you saw in Figure 5. A single incoming message could be processed by a handful of different orchestrations and send ports. Or you could have a handful of different receive locations (each bound to a different adapter/transport) that receive the same logical message and execute the same orchestration.

Subscriptions are basically filter expressions that return a Boolean result when evaluated. You can use logical ANDs and ORs to group multiple filter expressions together. Figure 6 shows an example of two filter expressions—one that matches the message type (the namespace name plus the local name) and another that checks a value from the message body. Each filter expression is evaluated against the incoming message's context, which can contain a variety of information. It typically contains some BizTalk-specific information (such as port details) along with information propagated from the transport, and any properties promoted from the message body itself.

Figure 6 Defining Filter Expressions (Subscriptions)
Figure 6 Defining Filter Expressions (Subscriptions)

Developers promote nodes in the message schema that they want to make visible to the message context. Figure 7 illustrates how this is done using the Promote Properties dialog accessible from the BizTalk Schema Editor. Once promoted, information about the properties is stored in the schema definition so BizTalk knows what to promote when messages arrive at run time. The promotion process occurs after a message is received by a port but before it's persisted to the MessageBox.

Figure 7 Promoting Properties
Figure 7 Promoting Properties

You can think of promoted properties as a view over the entire message as in a SQL view (see Figure 8). This view only needs to contain the information that you need for routing and decision purposes. This makes it possible for BizTalk Server to optimize message processing, which is helpful with large messages.

Figure 8 Message Properties
Figure 8 Message Properties

This publish-subscribe mechanism makes the messaging layer asynchronous by nature. Synchronous request/response style operations are possible but they're still implemented around the centralized MessageBox in an asynchronous way, which means you'll never achieve the same speeds as traditional synchronous communications. The publish-subscribe mechanism decouples senders from receivers and allows new participants to be added at any point without affecting the existing ones. It also enables some sophisticated content-based routing possibilities through promoted properties.

Contract-Driven Messaging

Now that I've covered the major pieces of the messaging architecture, let's review the high-level development process. Developers typically start by modeling the messages that will flow through their system across the various applications. These messages are defined using XML Schema via the BizTalk Schema Editor. Developers then promote the message properties that will be used for routing or decision purposes. The promotions found in the schema-based contracts are then used by the messaging components to populate the message context at run time and evaluate message subscriptions. Developers can then configure send ports or orchestrations to subscribe to messages matching criteria related to these properties.

As you can see, the entire development process starts with the original schema definitions and everything builds on that. The schema-based contracts dictate how messages are processed, how subscriptions are evaluated, and ultimately how messages flow through the system. They really lay the foundation for the entire solution. It's truly an example of contract-driven messaging at its finest.

Port Extensibility

Throughout this article, I've focused on the BizTalk Server 2004 approach to XML messaging and some of the basic architectural principles that make it all work. I've also highlighted various aspects of the BizTalk Server 2004 architecture that have resonated with me as an XML developer. But this is only the tip of the iceberg.

Figure 9 Port Internals
Figure 9 Port Internals

In addition to what I've covered here, BizTalk Server 2004 provides some powerful extensibility points that allow you to perform additional processing while a message is being received but before it's persisted to the MessageBox (see Figure 9). It also provides the same extensibility points when messages leave the MessageBox. The three port extensibility points are adapters, where bytes are read; pipelines, where preprocessing is performed in stages; and maps, where the message can be transformed, as shown in Figure 9. XML developers will find these extensibility points particularly interesting due to the flexibility they introduce and the common problems they help solve. I'll cover port internals in detail in the next installment of this column.

Wrap-Up

BizTalk Server has changed dramatically over the years, maturing from its humble beginnings into something more useful and applicable than many developers imagined possible. The current version has many gems for the XML developer better tools and automated support for some of the most common XML integration tasks. BizTalk Server provides a message-oriented approach to connecting systems, something that's truly service-oriented by design. In Part 2 of this series, I'll dive into the port extensibility points including adapters, pipelines, and maps. But while you're waiting, check out "Understanding BizTalk Server 2004", "BizTalk Server 2004: A Messaging Engine Overview", and Pluralsight's BizTalk Wiki.

Send your questions and comments for Aaron to sstation@microsoft.com.

Aaron Skonnard is a co-founder of Pluralsight, a Microsoft .NET training provider. Aaron is the author of Pluralsight's Applied Web Services, Applied BizTalk Server 2004, and Introducing Indigo courses. Aaron has spent years developing courses, speaking at conferences, and teaching professional developers. Reach him at pluralsight.com/aaron.


Bugslayer: SUPERASSERT Goes .NET

Bugslayer
SUPERASSERT Goes .NET
John Robbins

Code download available at: Bugslayer0511.exe (221KB)
Contents
Requirements
Using SUPERASSERT.NET
Implementation Details
End Notes

Those of you who have been reading this old Bugslayer column over the last nine years have branded into your frontal lobe a single word: ASSERT! Anytime you can have the code tell you about a problem instead of having to find it by slaving away with a debugger is a huge timesaver. As you'll remember from your native C++ days, you had to do all your own proactive programming from scratch. In the Microsoft® .NET Framework world, the heavy lifting is done by the Base Class Library's (BCL) Debug.Assert method. Add to that the outstanding extensibility in the TraceListeners and you've got a debugging system that's ready to handle just about anything.

Even though there's such great debugging support, it's not perfect. When a .NET assertion is triggered, you can't see the parameters in the stack trace and you can't look at the call stacks of other threads in the process. But when you consider the sandboxing and security that .NET offers, you can see why the system hides that information. If you could get parameters and call stacks from other threads, you'd be right back in the wild days of native C++.

That said, there are still times when you really need to see those other call stacks and parameters. Think about the times you can't duplicate a bug on your development machine and you need to be running on a test lab machine. If you could look at the data and snap a mini dump of the app at assertion time, you'd almost give up your 1993 PDC Slinky or something equally significant.

What's needed for .NET is the equivalent of the native C++ SUPERASSERT that I wrote way back in 1999 when this magazine was Microsoft Systems Journal (see Bugslayer - February 1999), and extended in my book Debugging Applications for Microsoft .NET and Microsoft Windows (Microsoft Press®, 2003). Not only did SUPERASSERT walk the stack for any thread (along with all parameters and locals), it had mini dump support, extra cool ignore options, and even allowed you to e-mail the assertion. SUPERASSERT was supposed to be the last assertion code you would ever need.

Many of you have asked if it would be possible to do SUPERASSERT in the managed world. I had my doubts, but I've come up with something close, which I'll present here.

Requirements

Let's begin with the requirements. I needed to be able to walk the stack for other threads in the process and write a mini dump when the assertion UI was shown—some pretty serious requirements.

I wanted something very similar to the user interface from the native SUPERASSERT I'd done for my book because it has worked well. I know that testers and other developers need a way to mark an assertion as ignored so it does not keep popping up. For example, if you've got normal error handling reporting to the user that the disk is full, having testers and developers fight through three or four assertion displays to get to the normal error is extremely annoying. Plus, they may need to turn off all assertions for automated tests.

Another key requirement is the ability to copy out of the assertion UI to paste into a bug report. Equally important is the option to e-mail the assertion to the developer who is responsible.

The ability to jump into the debugger when necessary is also critical. Debugging in .NET means debugging managed and native code. Visual Studio® .NET is more than sufficient for most of your managed debugging needs, but if you're using native DLLs or COM components, or have to see odd managed memory problems, you need the native debugging power of tools such as Son of Strike (SOS), which I've talked about before (see SOS: It's Not Just an ABBA Song Anymore and Mini Dump Snapshots and the New SOS). One requirement I had in the native C++ version of SUPERASSERT was the option to suspend all other threads in the process from the asserting thread. That allowed you to avoid synchronization issues whenever an assertion triggered. While it's possible to suspend threads in the .NET version, there's the small problem of garbage and finalizer threads. If I would suspend threads, I'd kill the memory management system in .NET and that's certainly not a good idea.

Finally, I wanted the code to work on both Win32® and Win64, but be able to run as a 32-bit application on Win32 and a 64-bit application on Win64.

Using SUPERASSERT.NET

When you're talking about using Debug.Assert and creating new output, you're really looking at a custom TraceListener. In fact, SuperAssertTraceListener is derived from DefaultTraceListener, so I only had to override the Fail methods.

It's pretty simple to meet all my requirements for free (almost). First, copy to your application's location the three DLLs that make up the utility (Bugslayer.DLL, BugslayerUtil.SuperAssertSettings.dll, and BugslayerUtil.SuperAssertSettings.XmlSerializers.dll). Next, add SuperAssertTraceListener from Bugslayer.DLL to the Debug.Listeners collection. You can do that either programmatically or by using TraceListeners in your app.config.

This last step is a little more interesting and you'll see why this is required when I talk about the implementation. You'll need to install the Debugging Tools for Windows on at least one machine. For test machines, or production if you are so inclined, you just need to XCOPY the Debugging Tools for Windows directory to the machine.

After you have an assertion in your application, you'll see the assertion message in Figure 1. The edit control at the top of the window shows the message and detailed message parameters you passed to Debug.Assert. You also see the module, source, and line where the assertion occurred. So far, it's the same information you'd see in the standard DefaultTraceListener.

Figure 1 SUPERASSERT .NET Main Dialog
Figure 1 SUPERASSERT .NET Main Dialog

The LastError value shows you the last native Windows error as reported by GetLastError, which can be helpful, especially if you're doing lots of interop. Note that the value displayed here might not have anything to do with the assertion you're seeing. My code saves off the last error value as soon as it's called. However, the last error value could have been changed by a previous TraceListener in the Listener collection.

The penultimate value in the edit control is the number of times this particular Debug.Assert failed. The last value in the edit control is the number of native Window handles your process currently has open. Leaking handles is a huge problem in both native and managed code so seeing the total number of handles can, in fact, help you detect potential problems.

I'm sure you can guess what the Ignore Once button does for the current assertion. The Abort Program button is a true death button, as it will call the Windows TerminateProcess function to rip the process completly out of memory. You'll want to be extra careful pressing that button as there's no turning back.

The Managed Debugger button triggers the managed debugger. If you are debugging the process, it will call Debugger.Break to stop in the debugger. If there is no debugger present, SUPERASSERT.NET will call Debugger.Launch to start the just-in-time debugger process so you can choose the debugger to use. As you would expect, you need to have sufficient operating system privileges to debug the process.

The Copy To Clipboard button copies the values from all textboxes in the dialog to the clipboard. Don't be surprised if the button is disabled in your application. It turns out that the .NET clipboard class relies on COM, so if your main thread is not marked as a single-threaded apartment (STA) thread, you'll get interesting exceptions attempting to use the clipboard classes. There are a few solutions to this problem. One is to spin up a new STA thread and do all of the clipboard interaction on that thread. Another is to use interop to access the Windows clipboard API. In a future release, I'll do the latter.

The Create Minidump button brings up a simple dialog where you can specify the name of the mini dump you want to write. As with the clipboard classes, the .NET FileBrowserDialog class uses COM so it's not safe to use without an STA main thread and, as a result, the dialog you'll see is nothing fancy. However, the dialog will make sure the path to the file exists and give you a simple check option to automatically put the date and timestamp on the file name to ensure the mini dump file is unique.

The mini dumps that I created are the appropriate full-memory mini dumps, which means you can feel free to bang away at them with SOS and really see what's going on in your application. That also means they can get huge. For the simple test program, I created full-memory mini dumps on Windows XP Professional x64 Edition that were 421MB!

The e-mail assertion will only be active if you've made a very small change to your code. In order to identify who owns the code, you'll need to add the CodeOwner attribute from the Bugslayer.Diagnostics namespace in BugslayerUtil.DLL to your classes, as shown in the following snippet:

[CodeOwner ( "John Robbins" ,
   "john@wintellect.com" )]
class Program
...
SUPERASSERT.NET will look up the stack of the assertion for the first class with the CodeOwner attribute and use that e-mail address as the To field for the mail. When using the CodeOwner attribute, please make sure to remove my e-mail address because, as you can imagine, I just don't need any more mail. The e-mail code in SUPERASSERT.NET uses the new SmtpClient class from the System.Net.Mail namespace.

When you press the Email Assertion button, you'll see the window in Figure 2, which has the text from the main window in the body portion and the To and Subject fields already set. The default operation is to use the mailSettings element in the app.config or machine.config. If you're working on an application where your e-mailed assertions will be coming from inside your firewall, you can set up a hardcoded account with user names and passwords in the app.config so that e-mailing assertions is trivial.

Figure 2 E-Mail the Assertion
Figure 2 E-Mail the Assertion

If you're working on an application in which people outside your firewall will be sending messages, you'll want to tell them to deselect the Use default SMTP credentials from app.config. When they uncheck that box, the SMTP Settings button is enabled and they can enter in their username, host, port, and secure sockets layer (SSL) settings for their SMPT server. SUPERASSERT.NET saves those settings so the user doesn't have to enter them again. By the way, all SUPERASSERT.NET settings are being stored in C:\Documents and Settings\<user>\Local Settings\Application Data\Bugslayer\SuperAssertTraceListener\1.0.0.0\Settings.Config so they are global to all processes on the machine.

When users set their own SMTP settings, they'll be prompted for their password. The SUPERASSERT.NET code does not store that password, but it does appear in unencrypted form in memory briefly, so it may be a security violation depending on your company's security standards.

Overall, I'm not so happy with the SMTP approach for e-mailing assertions because it's not as seamless as it should be. Maybe in SUPERASSERT.NET 2.0, I'll write a little plug-in architecture that allows you to use the e-mail program on the user's machine so it's a better experience. Note that if you're using Beta 2, the SmtpClient code does not work with Google's GMail as your SMTP server. This is expected to be fixed in the final release.

The More>> button in Figure 1 is pretty exciting. When you click on it, the dialog expands to look like Figure 3. Since I don't do much UI development, I expect you to shout ooh and aah every time you click on the More>> and Less<< buttons. It's also fun to click on those buttons very fast.

Figure 3 Expanded SUPERASSERT.NET
Figure 3 Expanded SUPERASSERT.NET

The Ignore group contains advanced options for disabling specific assertions or even all assertions, and should be used with care. The Stack group allows you to walk the stacks of all the other managed threads in the application. The default is to walk only the managed stack that had the assertion. The Thread ID combobox allows you to look at the call stacks from other threads. As a byproduct of the method I'm using to get the other managed thread stacks, you won't see the source file name and line for the other threads, only the thread with the assertion.

The last group box, Native Debuggers for SOS, lets you choose either WinDBG or CDB (Console Debugger) to do the hardcore, ninja-level debugging. When SUPERASSERT.NET spawns the debuggers it automatically loads the appropriate version of SOS so you can immediately start hacking at the managed heap or stack with no slowdowns.

Figure 4 Options Dialog
Figure 4 Options Dialog

In past columns, I've discussed WinDBG, but you may have never heard of CDB. Both debuggers are essentially the same. WinDBG has a GUI (and I use that term very loosely) and CDB is console-based. The Command window in WinDBG and CDB itself are just wrappers on the same DLL—DBGENG.DLL.

The last piece of the UI I'll mention is the Options dialog, which is accessible through the Options menu item on the system menu and is shown in Figure 4. You can figure out what the options do just by looking at the dialog, which is set to the defaults. You'll probably want to leave the stack-walking set to initially show only the thread that had the assertion because walking the other threads is slow.

Also, SUPERASSERT.NET is smart about its z-order. If you're running under any flavor of debugger, it knows not to set the TopMost property because doing that will make the SUPERASSERT.NET window appear on top of the debugger, making debugging impossible. Any options you set in this dialog are also saved to the settings file I mentioned earlier.

Implementation Details

Implementing SUPERASSERT.NET was what you might call "fun," if you're so inclined. Although I wasn't able to get the parameters and locals of items on the stack directly in the SUPERASSERT.NET dialog, I was able to get a glance at other thread stacks from an assertion and write correct mini dumps that can be debugged at a later time.

On any project, I keep a list of problems I expect to run into before I start coding. An important problem with SUPERASSERT.NET was how to ignore and skip individual assertions. From a StackTrace object, you can access the individual items on the class with their StackFrame class. The stack frame has the Type, which gives you the fully qualified name of the class, the method name, and the intermediate language (IL) offset into the method. Combine those three items and you've got a unique hash key to associate with an object that contains fail and ignore counts.

The number-one item on my tough problems list was how to walk the other threads and create a correct mini dump. I'd noticed in the .NET Framework 2.0 documentation that the StackTrace class has a new constructor that takes a Thread object. The idea is that you can pass in any Thread object and get that thread's stack. That gave me some hope of enumerating the threads from inside the application and getting their corresponding Thread objects.

You may have noticed that the Process.Threads property is a ProcessThreadCollection that contains the ProcessThread objects for all the native threads in the process. Thinking that there might be a way to convert those ProcessThread objects into the equivalent Thread object, I spent some serious quality time with the compiler and Reflector trying and looking at ideas. After pounding through everything I could think of, I realized that the only way I was going to get the information from other stacks was to spawn off a process that would pass that information back into SUPERASSERT.NET. I pulled up the common language runtime (CLR) Debugging API headers and suddenly realized that I didn't need to write my own debugger to get the stacks, I already had one on the computer, CDB.EXE paired with a little SOS pixie dust in the form of ~*e!clrstack.

The ~*e command in CDB/WinDBG says to run the following for all threads in the process. The command, !clrstack, is the SOS command to walk the managed stack. All I needed to do was spawn off CDB, turn on logging in CDB, run the ~*e!clrstack, close the log, and parse the results. While the idea would be to walk all the stacks from inside the process, there's no viable way to make that happen. This is a fair trade-off, especially considering that you just need to XCOPY the Debugging Tools for Windows directory to the target machine. While I haven't tried it, you may be able to cherry pick binaries out of the Debugging Tools for Windows directory and still have SUPERASSERT.NET run perfectly for you.

Some of you might be wondering if using CDB like this will work because of processes that are not running with administrator privileges or the SeDebugPrivilege. The SeDebugPrivilege is important only if you are going across user accounts on the machine. The good news is that SUPERASSERT.NET is spawning the CDB process on itself, so as long as the process has permissions to see and run CDB on the disk, this approach will work.

As soon as I tried my idea, I of course ran into a problem. I was starting CDB with the -c command line, which allows you to pass the commands to execute when the debugger starts. I was passing in a string with multiple commands separated by the standard semicolons to indicate CR/LF. Unfortunately, CDB randomly skips some of the commands in the string. One run would have the log file opened and the next would not. I switched to putting the commands in a text file, with one command per line and using the —cf option to tell CDB to read the commands out of the script file. That worked so I could get the call stacks.

The script format string contains the following CDB commands:

.loadby sos mscorwks
~*e!clrstack
.logopen "{0}"
~*e!clrstack
q

The first line tells CDB to load the SOS.DLL file that's in the same directory as the core CLR file, MSCORWKS.DLL. The second line walks all the stacks. You see two stack walks in the script. The reason is that to do the stack walking, !clrstack forces appropriate PDB source files to load, and CDB (being the output spew monster) interjects lots of messages about what loaded and didn't. Because those messages are random, parsing is drastically harder, so I just let !clrstack run and CDB can spew all it wants there. After opening the log file, which is actually in the temp directory, I do the real ~*e!clrstack that's just the output from SOS.

The command-line format string I pass to CDB.EXE is: -p {0} -pv -cf "{1}". The -p is the process ID to attach. The -pv is a neat trick in that it tells CDB.EXE to do a noninvasive attach. That way you are not actually debugging the application. On Windows 2000, if CDB, which is a native-only debugger, attached normally to the process, as soon as the debugger ended, the .NET process that spawned CDB.EXE would end because there's no support for native detaching. The —cf is the script file to run. Like the output file, the script file is written to the temporary directory.

As soon as CDB ends, I pick up the output file and parse out just the managed stacks using the following regular expression string:

const String regExtractClrStackPattern =
     @"(?# Get the thread ID for this block into the TID group)"+
     @"(Id:\s(?<TID>0x[0-9a-fA-F]+)\s\([0-9]+\)\r\n+)"+
     @"(?# Skip over the header pieces for x64 or x86)"+
     @"(?:^(Child|ESP).+\r\n)"+
     @"(?# Grab the stack into the CALLSTACK group)"+
     @"(?<CALLSTACK>(^[0-9a-fA-F]+\s[0-9a-fA-F]+\s.+\r\n)+)";
You can read the comments embedded in the expression itself to see what's going on.

After pulling out just the managed call stacks from the output, I remove the stack pointer and return addresses from the strings. After that, it's just a matter of storing those call stack strings so I can show them when requested by the user.

One set of data I wanted to add to the call stacks was parameter and local values. You might be wondering why I didn't take advantage of the -a option to !clrstack, which dumps locals and parameters. With the .NET Framework 1.1 versions of SOS.DLL, the output is good. Unfortunately, with the .NET Framework 2.0 Beta 2, the version of SOS.DLL I'm using has some problems with locals and parameters (it shows some, but not others). This problem is supposed to be fixed by the release of the .NET Framework 2.0. I'll update SUPERASSERT.NET to take advantage of the fixed version of SOS.DLL. Since I'm using CDB to do the stack walking, you can probably guess how I'm handling the mini dump creation from SUPERASSERT.NET.

End Notes

For those of you who were hoping to see some serious work to get consistent correct call stacks from other threads from inside a managed application, I'm sorry to disappoint. While it may be doable, it would be extremely fragile and hard to fully support on multiple platforms. Overall, I'm happy with the approach of having CDB take care of getting stacks and mini dumps because it works and it's maintainable.

The approach I took also produces SUPERASSERT.NET, which in the end is the most important outcome. Now you can get an idea of the state of your application during an assertion. Better than that, you can get the mini dump for later and even get a native debugger with SOS all over the process when you want as well. The idea is anything that can help you debug faster is a great thing!

Tip 71 SUPERASSERT.NET is a TraceListener. Another fantastic example of a TraceListener is Josh Einstein's TerminalTraceListener. TerminalTraceListener opens a port and allows you to connect to your app through Telnet so you can watch your traces in the lab or even across the globe. I wish I would have thought of TerminalTraceListener!

Figure 5 Keyboard File
Figure 5 Keyboard File
Tip 72 Peter Provost showed me an amazing undocumented trick in Visual Studio .NET for all the total keyboard people out there. If you go to the find combobox (Ctrl+D with the default keyboard mapping) and you type "> open" (without quotes). The ">" flips the find combobox into a mini Command window. The "open" part is the aliased command to the "file open" action. The super part is when you press space and the first character of the file you want to open anywhere in your solution, a dropdown window pops up with the files and folders that start with that letter. See Figure 5 for the magic.

Send your questions and comments for John to slayer@microsoft.com.

John Robbins is a cofounder of Wintellect, a software consulting, education, and development firm that specializes in .NET and Windows. His latest book is Debugging Applications for Microsoft .NET and Microsoft Windows (Microsoft Press, 2003). You can contact John at www.wintellect.com.


Security Briefs: Security Features in WSE 3.0

Security Briefs
Security Features in WSE 3.0
Keith Brown

Code download available at: SecurityBriefs0511.exe (116KB)
Contents
Here Comes WSE 3.0
Turnkey Security Assertions
Using a Turnkey Assertion via Policy
Turnkey Assertion via Code
Custom Policy Assertions
State Management
Conclusion

I've been spending a lot of time lately building secure Web services with the Microsoft® .NET Framework 2.0, and Web Services Enhancements (WSE) 3.0 has been a lifesaver for me, so I thought it would be appropriate to dedicate a column to security features in this new product. This is the first version that has been built around Visual Studio® 2005, and in fact WSE 3.0 won't run on version 1.1 of the .NET Framework, so all this information is still a bit cutting edge and subject to change by the final release. For this column, I'm using the Beta release of WSE 3.0.

Here Comes WSE 3.0

If you've used previous versions of WSE, you'll appreciate having the WSE settings tool built right into Visual Studio 2005, because it makes creating security policies a breeze. While this is similar to older versions of WSE, a number of things have changed. A set of turnkey security policy assertions takes a lot of the guesswork out of securing a Web service. If you have read my earlier article on the Username token (available at Securing the Username Token with WSE 2.0), you'll see firsthand how tricky it can be to build a bulletproof authentication system using WS-Security. Using a turnkey policy assertion will help you avoid many of the pitfalls mentioned in that article.

Building session-oriented systems has gotten a lot easier as well, with the stateful security context token (SCT) in WSE 3.0. By flipping a switch in your Web.config file, you can instruct WSE to keep enough state in its SCT that it can reestablish a WS-SecureConversation session with a remote Web service. With a stateful SCT, you get a measure of reliability (sessions can be restarted transparently if they are broken) plus support for sessions across Web farms.

In most Web service deployments, using a policy file to describe your security requirements is a great way to give the administrator the flexibility she needs to keep your Web service accessible and secure even when the security environment of the service changes. But in some cases, you might want to close this down to restrict the configuration options to the most basic requirements needed to support your application.

In previous versions of WSE, the policy model for specifying security configuration was divorced from the code-based model. In other words, if you didn't want to use a policy file as WSE defined it, you were forced to piece together the WS-Security header yourself. In WSE 3.0, you can create policy assertions in code, supplying the same parameters that you would normally supply via a policy file. This is a really important feature, especially given the complexity of implementing some of the new turnkey assertions. You wouldn't want to have to do it by hand!

Finally, one of my favorite features is the clean new model for building custom policy assertions. This is a great way to add custom authorization to a Web service, and it's gotten a lot easier in Web Services Enhancements 3.0.

Turnkey Security Assertions

Turnkey security assertions are the most important new security feature, so I'll focus on them first. The WSE team spent quite some time developing a number of strategies for securing Web services. They looked at server authentication with secure sockets layer (SSL) or WS-Security using an X.509 certificate, client authentication with either a certificate or a Username token, as well as other more specialized cases such as using Kerberos for mutual authentication and channel security.

Before I enumerate the strategies used by each of the turnkey assertions, I want to point out what they all have in common: each strategy seeks to provide authentication, message integrity, and confidentiality. However, each assertion approaches these security goals in different ways.

There are six turnkey assertions in the June CTP. The first is AnonymousOverCertificateSecurity, where the Web service authenticates itself to callers via its X.509 certificate. All callers are anonymous. Message-level integrity and confidentiality are supplied via a symmetric key that is exchanged using the server's public key. This model is logically similar to what you'd get if you surfed to your favorite online bookstore and purchased a book. The bookstore has too many customers to bother authenticating each of them, but it's happy to accept a credit card number and billing address, passing on the duty of authenticating the transaction to the bank.

In many business-to-business Web service scenarios, both the client and the server are assigned certificates. In this case, you would choose one of the following two assertions: CertificateMutualAuthenticationProfile or MutualCertificateSecurity. Logically these are very similar; both client and server authenticate by presenting their X.509 certificates to each other and prove possession of the corresponding private key. Message-level integrity and confidentiality are provided with a symmetric key exchanged using the server's public key. The first assertion relies on the existing WS-Security 1.0 spec, while the second uses the draft WS-Security 1.1 spec, which includes some new features such as encrypted SOAP headers.

While using certificates to authenticate clients is a great choice, often it's simply not feasible because of the overhead of maintaining the public key infrastructure (PKI) required for issuing and managing client certificates. Often it's more natural to authenticate clients with a simple user name and password. There are two turnkey assertions that support this: UsernameOverTransportSecurity and UsernameOverCertificateSecurity. You'd use UsernameOverTransportSecurity if you're relying on SSL to authenticate the server and provide integrity and confidentiality for the channel. UsernameOverCertificateSecurity is useful when SSL doesn't work for you (for example, when you have intermediaries and want end-to-end authentication).

And finally, if you're building Web services for an intranet where Kerberos is supported, which is the case in modern Windows® domain environments, the KerberosSecurity assertion is a very natural choice. Because Kerberos uses purely conventional cryptography to authenticate a client and service, it doesn't place nearly as heavy a load on the service when compared to certificate-based solutions. But the biggest benefit by far in a Windows domain environment is the support for single sign on. By using the client's default logon credentials to authenticate, there's no need to query the client for a password. And of course PKI isn't required either. The server can use Windows groups as roles to authorize access to resources or simply impersonate the client and let Windows handle the access checks.

Using a Turnkey Assertion via Policy

Using the WSE settings tool, it's easy to create a security policy with one of the turnkey assertions. If you installed WSE 3.0 with the Visual Developer option, you should see a WSE Settings 3.0 option at the bottom of your context menu when you right-click a project in the Solution Explorer. Selecting that will allow you to enable your Web service project for WSE. On the Policy tab, you can add a new policy using the security settings wizard. Each policy has a name, and you'll use that name to refer to your policy from code. When adding a new policy, you'll quickly come to the page shown in Figure 1. The authentication method you select here will determine which security policy assertion you end up with.

Figure 1 Choose an Authentication Method
Figure 1 Choose an Authentication Method

The wording in this dialog may confuse you at first, but when you choose an authentication method, you're deciding how the client will be authenticated. So if you wanted to use the AnonymousOverCertificateSecurity assertion that I just described, you'd pick the Anonymous option. Similarly, to use KerberosSecurity, you'd select the Windows option. The Certificate option leads you towards the two certificate-based mutual authentication assertions, and the final selection will depend on whether you choose to use WS-Security 1.1 extensions, a checkbox for which shows up later in the wizard. The Username option leads you towards one of the two Username assertions, and the final selection between transport and message-level security oddly enough occurs in the same spot: if you choose to enable WS-Security 1.1 extensions, you'll end up using the UsernameOverCertificateSecurity assertion, otherwise you'll end up with UsernameOverTransportSecurity. Look at the summary page in the wizard to confirm that you have the class of assertion you wanted. Figure 2 shows an example.

Figure 2 Summary Page
Figure 2 Summary Page

Depending on whether you're securing a service or client application, you'll often be given the choice of specifying credentials in the policy or specifying them via code. It's most convenient to specify service certificates via policy, you just need to ensure that the certificate is installed on the machine where your client or service will be deployed. On the other hand, the user name and password identifying a client should be set dynamically in code by calling SetClientCredential. Putting cleartext passwords in configuration files is a really bad idea.

A WSE policy file is simply a container of named policies. Once you've got your policies set up, you need to tell WSE which one to use. For a service, this is as easy as adding a PolicyAttribute to the class that implements the service. On the client side, I generally just call SetPolicy to choose the policy I want to use, although if you want to impress your friends, use a partial class on your proxy to add a PolicyAttribute. Figure 3 shows how I prefer to set policy. Note that you'll need a using statement for the Microsoft.Web.Services3 namespace to make your compiler happy with this sample code.

Figure 3 Wiring Policy for Services and Clients (Close Figure 3)
[Policy("MyServerPolicy")]
public class MyService : WebService {
    [WebMethod]
    public void DoSomething() {
        ...
    }
}

class ClientApp {
    static void Main(string[] args) {
        MyServiceWse proxy = GetProxy();
        proxy.DoSomething();
    }
    static MyServiceWse GetProxy() {
        MyServiceWse proxy = new MyServiceWse();
        proxy.SetPolicy("MyClientPolicy");
        return proxy;
    }
}

Turnkey Assertion via Code

It turns out that under the covers, when WSE initializes its security settings from a policy file, it actually creates an instance of a class that represents the policy assertion, then asks that instance to initialize its settings from the XML in your policy file. Once you know this, it's easy to skip that step and simply create the policy dynamically in your code. This technique lets you choose which settings you want an administrator to control at the time of deployment and which settings you want hardcoded. Of course you can shoot yourself in the foot if you hardcode too much configuration, so tread very carefully here.

A WSE policy is an ordered collection of policy assertions. Ordering is important when you have more than one assertion, as you'll see when I discuss custom policy assertions later. But for now I'll keep things simple and show you how to dynamically create a policy with a single turnkey security assertion.

Step one is to create the turnkey assertion and configure it. I've listed the classes representing these assertions below. They match the six turnkey assertions I described earlier:

  • AnonymousOverCertificateAssertion
  • CertificateMutualAuthenticationProfileAssertion
  • MutualCertificateAssertion
  • UsernameOverTransportAssertion
  • UsernameOverCertificateAssertion
  • KerberosAssertion

Imagine you wanted to use the popular UsernameOverCertificateAssertion. Figure 4 shows an XML policy that you might create using the wizard. This is a great way to learn to code against the assertion, as the attributes you see correspond directly to the properties on the assertion class. Figure 5 shows how you'd dynamically construct the equivalent assertion in code. If you look at these two figures side by side, you'll see how easy it is to go from one to the other.

Figure 5 Coding a Policy by Hand (Close Figure 5)
// here are the using statements required...
using System.Security.Cryptography.X509Certificates;
using Microsoft.Web.Services3.Design;
using Microsoft.Web.Services3.Security;

// and here is the code to construct the assertion...

UsernameOverCertificateAssertion a =
    new UsernameOverCertificateAssertion();

// usernameOverCertificateSecurity element
a.EstablishSecurityContext    = false;
a.RenewExpiredSecurityContext = true;
a.SignatureConfirmation       = false;
a.DeriveKeys                  = true;
a.ProtectionOrder = ProtectionOrder.SignBeforeEncrypting;

// serviceToken/x509 element
a.X509TokenProvider =
    new X509TokenProvider(StoreLocation.LocalMachine,
        StoreName.My, "CN=WSE2QuickStartServer",
        X509FindType.FindBySubjectDistinguishedName);

SignatureOptions so =
    SignatureOptions.IncludeAddressing |
    SignatureOptions.IncludeTimestamp  |
    SignatureOptions.IncludeSoapBody;

// protection/request element
a.Protection.Request.SignatureOptions  = so;
a.Protection.Request.EncryptBody       = true;

// protection/response element
a.Protection.Response.SignatureOptions = so;
a.Protection.Response.EncryptBody      = true;

// protection/fault element
a.Protection.Fault.SignatureOptions    = so;
a.Protection.Fault.EncryptBody         = false;
Figure 4 Sample Policy Configured in XML (Close Figure 4)
<policy name="MyServiceSecurityPolicy">
  <usernameOverCertificateSecurity
    establishSecurityContext="false"
    renewExpiredSecurityContext="true"
    signatureConfirmation="false"
    protectionOrder="SignBeforeEncrypting"
    deriveKeys="true"
    actor="">
  <serviceToken>
    <x509 storeLocation="LocalMachine"
        storeName="My"
        findValue="CN=WSE2QuickStartServer"
        findType="FindBySubjectDistinguishedName" />
  </serviceToken>
    <protection>
      <request
        signatureOptions="IncludeAddressing, IncludeTimestamp, IncludeSoapBody"
        encryptBody="true" />
      <response
        signatureOptions="IncludeAddressing, IncludeTimestamp, IncludeSoapBody"
        encryptBody="true" />
      <fault
        signatureOptions="IncludeAddressing, IncludeTimestamp, IncludeSoapBody"
        encryptBody="false" />
    </protection>
  </usernameOverCertificateSecurity>
</policy>

Now that you've got a policy assertion, you need to wrap it in a Policy object so that WSE can use it. Creating a policy is a no-brainer— just do this:

// create a policy, then add in
// our assertion
Policy policy = new Policy();
Policy.Assertions.Add(a);

The last step is to wire up the policy to your service or client proxy. To wire it to a client proxy, just call SetPolicy as I did in Figure 3, passing the policy object instead of a string. SetPolicy is overloaded to accept either a string representing a named policy in an XML policy file or a dynamically constructed Policy object.

Wiring up a service is a little bit different. You'll need to create a new class that derives from WSE's Policy class, use your constructor to build the assertion, then add it to your Assertions collection. Here's an example:

public class MyServerSecurityPolicy : Policy {
    public Policy() {
        ... // construct assertion here
        Assertions.Add(a);
    }
}

Now you can use the PolicyAttribute just like before to indicate which policy to use for your service, only this time you'll specify the policy as a type instead of a string:

[Policy(typeof(MyServerSecurityPolicy))]
public class MyService : WebService {
    [WebMethod]
    public void DoSomething() {
        ...
    }
}

Custom Policy Assertions

As I said, a policy is just an ordered collection of policy assertions. Those policy assertions are actually instantiated when WSE is initializing a service or proxy at run time. Each assertion is then asked to provide code for implementing its particular policy in the WSE message-processing pipeline. So think of policy assertions as templates for laying out the pipeline that will process each message; the policy determines the order steps will execute.

When I wrote the WSE security hands-on lab, I built a logging security assertion that could be placed anywhere in the pipeline. For example, if you wanted to see what an outgoing message looked like before the security assertion encrypted it, you'd order the policy so that the logging assertion came before the turnkey security assertion. Heck, if you also wanted to see what the message looked like after it was encrypted, you could place another instance of the logging assertion after the security assertion, and just point it to another file. Here's what the XML for this policy might look like (I've omitted the details of the security policy assertion):

<policy name="MyServicePolicy">
  <myTrace output="beforeSecurity.xml"/>
  <usernameOverX509Security ... />
  <myTrace output="afterSecurity.xml"/>
</policy>

Since WSE has no idea what the <myTrace> element is for, I had to tell it the name of the corresponding policy assertion class that I wrote to handle logging:

<extensions>
  <extension name="myTrace" type="MyAssertion, MyAssembly"/>
  <!-- omitted the standard WSE entries -->
</extensions>
This simple assertion is not only a great way to help debug a Web service using WSE 3.0, but it's also a good example to follow if you want to try writing your own custom assertions.

So why am I bothering to talk about this feature in a column dedicated to security? Policy assertions are a great way to hoist common code out of a Web service, or even a set of Web services. Performing security checks in the WSE pipeline before the message actually reaches the Web method is an easy way to help application developers who might otherwise forget to make those security checks on their own.

There are a lot of security-related features that you can pull up into the pipeline like this. For example, if clients use certificates to authenticate to your service, you might want to map those certificates onto real Windows user accounts so that you can make requests to back-end servers using Kerberos credentials. Role-based access checks are another example. Any security feature you can pull up into the pipeline is a good thing, as it helps ensure a consistent security policy will be applied across all your Web methods.

What about validation of requests? Validating user input is an important step in building any secure system. If you can hoist some of that up into the pipeline, more power to you. Aaron Skonnard did some work in this space for WSE 2.0, so if you're interested in seeing an example, read the XML Files column in the March 2004 issue at WS-Policy and WSE 2.0 Assertion Handlers). If you think you might want to build your own custom policy assertion, start with my trace assertion and work from there, but let me leave you with some hints that will help your process.

Each policy assertion gets a chance to inject objects called filters into the pipeline that actually implement the policy assertion. The filter architecture is really simple. You must derive a class from SoapFilter and override one method: ProcessMessage. The following is an example of a filter that just lets the message flow through without doing anything:

class NoOpFilter : SoapFilter {
    public override SoapFilterResult ProcessMessage(
            SoapEnvelope envelope) {
        return SoapFilterResult.Continue;
    }
}

There are four separate pipelines in WSE:

  • The sender's output pipeline
  • The receiver's input pipeline
  • The receiver's output pipeline
  • The sender's input pipeline

Note that the order I listed these pipelines follows the natural flow of a request/response message pair from the sender to the receiver and back to the sender. Because a policy assertion might need to behave differently in each of these four spots, the policy assertion is asked to provide filters for all four locations. In fact, you override four methods in your policy assertion class to supply the following filters:

  • CreateClientOutputFilter
  • CreateServiceInputFilter
  • CreateServiceOutputFilter
  • CreateClientInputFilter

Note that if your policy assertion is only meant to run in the service process, you can simply throw an exception if you are asked to create a filter for a client pipeline, because the client functions will never be called for a service pipeline. The same holds true for a client-side assertion.

In my logging policy assertion, all four filters do the exact same thing: just log the current message to a named flat file. So in my policy assertion I simply hand out an instance of the same filter class for all four methods. Additionally, I override the ReadXml method on my policy assertion so that I can configure the name of the file for each of the filters based on attributes in the XML policy file. I pass this value along to the filter I'm creating.

Things start to get interesting when you construct a policy with multiple assertions. Remember that policy determines the order of processing in the pipeline. Imagine you write the following policy and use it for both your client and your service:

<policy name="MyPolicy">
    <assertionOne/>
    <assertionTwo/>
</policy>
Figure 6 Assertions Map to Filters
Figure 6 Assertions Map to Filters
Because assertionOne is listed first, it'll be asked to inject its filters first. Then assertionTwo will inject its filters. The resulting pipelines for the client and service will be ordered as shown in Figure 6, with the order of processing for a typical request/response message pair indicated in the diagram.

Here's how I remember this ordering—messages flow between your program and the wire. Consider your program to be at the top of the policy, and the wire to be at the bottom. It works this way on both the client and server side, making it possible to share policy files between client and server without having to reorder them.

State Management

Sometimes you will want your filters to be stateful, either for the duration of a single request/response pair, or perhaps across several different requests. WSE provides three state bags that you can use for this purpose. MessageState allows state to be shared for the duration of a message as it flows through a single pipeline. OperationState is designed for messages that are paired into request/response pairs: you can write OperationState during the request and read it back during the response. And finally, SessionState, which is managed on a per-proxy basis, allows you to store state across requests.

All of these state bags are available to you via properties of the SoapContext class, which itself is surfaced via the Context property of the SoapEnvelope class. If you look back at my filter example, you'll notice that the SoapEnvelope is passed to your filter, so these state bags are easily accessible.

Conclusion

While developing the logging assertion, I was impressed with the logic the WSE 3.0 pipeline architecture displays. Even without documentation I was able to get a simple logging assertion up and running in about 30 minutes. Thanks to the WSE team for a job well done! My experience shows that WSE 3.0 features will be appreciated by novice and advanced developers alike. For me, the custom policy assertion architecture gives me the flexibility I need to build security plumbing that developers can then easily reuse in their applications via policy. If you are new to Web services, the turnkey security policy assertions in WSE 3.0 will help you build systems using best practices and avoid many of the subtle pitfalls inherent in Web service security. You'll also be well prepared for Windows Communication Foundation (WCF) because WSE 3.0 was designed to be wire-compatible with WCF (formerly code-named "Indigo"). For more information on WSE 3.0, see Niels Flensted-Jensen's article on WSE and Authorization Manager in this issue, as well as Mark Fussell's WSE 3.0 overview article at What's New in Web Services Enhancements (WSE) 3.0.

Send your questions or comments for Keith to briefs@microsoft.com.

Keith Brown is a co-founder of Pluralsight, a Microsoft .NET training provider. Keith is the author of Pluralsight's Applied .NET Security course, as well as several books, including The .NET Developer's Guide to Windows Security, which is available both in print and on the Web. Learn more at www.pluralsight.com/keith.


C++ at Work: Disabling Print Screen, Calling Derived Destructors, and More

C++ at Work
Disabling Print Screen, Calling Derived Destructors, and More
Paul DiLascia

Code download available at: CatWork0511.exe (176KB)

Q I'm building an application that displays proprietary graphics. Is there some way I can disable the Print Screen function to prevent users from copying images to the clipboard?

Martin Cruz

A There is a way to disable Print Screen, but I should warn you that there's no way to prevent another application from grabbing the pixels from your window. Many third-party programs are available that can capture the screen, and it's also easy to write such a program. All you have to do to capture the pixels on the screen is copy them from the screen device context using BitBlt. For example:

CWindowDC dc(NULL);   // use NULL to get whole screen
CDC memdc;
... // create, initialize memdc
memdc.BitBlt(..., &dc); // copy screen contents

To copy the active window, you'd construct your CWindowDC from a pointer to the CWnd whose pixels you want to grab.

So the bottom line is there's no way you can prevent an application from capturing your window's pixels. That said, if all you want to do is disable Print Screen, or intercept it to do something else, it's fairly easy. Windows implements Print Screen using a registered hotkey. In my December 2000 column, I showed how to use RegisterHotKey to register a hot key for your app (see C++ Q&A: Sending Messages in Windows, Adding Hot Keys to your Application). Windows uses the predefined hotkeys IDHOT_SNAPDESKTOP and IDHOT_SNAPWINDOW to handle Print Screen. These correspond to Print Screen, which captures the entire screen, and Alt+Print Screen, which captures only the active window. To disable these functions all you have to do is register the hotkeys, which causes Windows to send your app a WM_HOTKEY message when the user presses either hotkey. Your implementation can ignore the message to bypass the default screen-capture behavior. A good place to do it is in your mainframe class.

Figure 1 shows the code for a typical MFC CMainFrame class implemented with MFC. The OnCreate/OnDestroy handlers register and unregister the hotkey IDHOT_SNAPDESKTOP; the OnActivate handler registers and unregisters IDHOT_SNAPWINDOW when the app is activated or deactivated. By reenabling IDHOT_SNAPWINDOW when your window is inactive, users can still use Alt+Print Screen when a different app has focus.

Figure 1 Handling Hotkeys (Close Figure 1)

MainFrame.h
#include "FolderFrame.h"
#include "resource.h"

////////////////
// Typical MFC Main frame window, override to disable PrintScreen.
//
class CMainFrame : public CFrameWnd {
protected:
...
   afx_msg int  OnCreate(LPCREATESTRUCT lpCreateStruct);

   // disable PrintScreen
   afx_msg void OnActivate(UINT nState, CWnd* pWndOther, BOOL bMinimized);
   afx_msg LRESULT OnHotKey(WPARAM wp, LPARAM lp);
   afx_msg void OnDestroy();

   DECLARE_MESSAGE_MAP()
};
MainFrame.cpp
#include "StdAfx.h"
#include "MainFrm.h"
#include "View.h"

IMPLEMENT_DYNCREATE(CMainFrame, CFrameWnd)
BEGIN_MESSAGE_MAP(CMainFrame, CFrameWnd)
...
   // disable PrintScreen:
   ON_WM_CREATE()
   ON_WM_DESTROY()
   ON_WM_ACTIVATE()
   ON_MESSAGE(WM_HOTKEY, OnHotKey)
END_MESSAGE_MAP()
...
int CMainFrame::OnCreate(LPCREATESTRUCT lpCreateStruct)
{
...
   RegisterHotKey(m_hWnd, IDHOT_SNAPDESKTOP, 0, VK_SNAPSHOT);
   return 0;
}

void CMainFrame::OnDestroy()
{
   UnregisterHotKey(m_hWnd, IDHOT_SNAPDESKTOP);
}

//////////////////
// Handle hotkey: should be PrintScreen or Alt-PrintScreen.
// Do nothing (bypass Windows screen capture)
//
LRESULT CMainFrame::OnHotKey(WPARAM wp, LPARAM)
{
   UNREFERENCED_PARAMETER(wp);
   return 0; // ignore
}

//////////////////
// When window is activated/deactivated, disable/enable Alt-PrintScreen.
// (IDHOT_SNAPWINDOW)
//
void CMainFrame::OnActivate(UINT nState, CWnd* pWndOther,
   BOOL bMinimized)
{
   CFrameWnd::OnActivate(nState, pWndOther, bMinimized);
   if (nState)
      RegisterHotKey(m_hWnd, IDHOT_SNAPWINDOW, MOD_ALT, VK_SNAPSHOT);
   else
      UnregisterHotKey(m_hWnd, IDHOT_SNAPWINDOW);
}

You might think you could prevent screen capture by registering your window class using the style CS_OWNDC (which causes Windows to allocate a private device context (DC) for your window class) but that won't work. Windows still copies the bits from your window's private DC to the screen DC, so any app that accesses the screen DC can see your pixels.

Q In your November 2004 column, you discussed calling virtual functions in managed and unmanaged code (see C++ Q&A: Calling Virtual Functions, Persisting View State, POD Type). Is it true that in Visual Studio® 2005, the destructor will be called for a derived class even if it's not declared virtual in the base class? In C++, if I want my derived class destructor to be called whenever the memory is deleted, I have to declare that destructor virtual in the base class.

Jigar Mehta

A The short answer is yes, if you're talking about managed classes. If the class is native, the standard C++ rules apply; if it's managed, the destructor is implicitly virtual.

The simplest way to understand what goes on with destructors is to write some code and see what the compiler does with it. Figure 2 shows a simple managed console app that declares two managed classes, CBase and CDerived. The constructors and destructors display printf diagnostics to show when they're called. If you compile this program using /clr, you'll see the following messages in your console window:

ctor: CBase
ctor: CDerived
dtor: CDerived
dtor: CBase

Figure 2 vdtor.cpp (Close Figure 2)
// To compile:
//
//    cl /clr vdtor.cpp
//
#include <tchar.h>
#include <stdio.h>

#using <mscorlib.dll>
using namespace System;

//////////////////
// Managed base class.
//
public __gc class CBase {
public:
   CBase()
   {
      printf("ctor: CBase\n");
   }

   // virtual keyword unnecessary for managed class because CBase is
   // implicitly derived from Object and the dtor is converted to a
   // Finalize method, which is virtual.

   /*virtual*/ ~CBase()
   {
      printf("dtor: CBase\n");
   }
};

//////////////////
// Managed derived class.
//
public __gc class CDerived : public CBase {
public:
   CDerived()
   {
      printf("ctor: CDerived\n");
   }
   ~CDerived()
   {
      printf("dtor: CDerived\n");
   }
};

//////////////////
// Program entry point.
//
int _tmain()
{
   // Create object: note pointer is declared as CBase* but actually
   // points to instance of derived class.
   CBase* pBase = new CDerived();

   // Explicitly delete to see which dtor is called...?
   delete pBase;

   return 0;
}

This shows that the derived destructor gets called even though neither the derived nor base class destructor is declared virtual. The constructors and destructors are called in the expected order, with the base class constructed first and destructed last.

Why don't you have to declare managed destructors virtual? Recall that every managed class must either be explicitly derived from another managed class, or else it's implicitly derived from the root base class, Object. The other thing you have to remember is that the C++ compiler converts your managed destructor to a Finalize method, which is virtual in the base Object class. To see this, all you have to do is examine the compiled code using the disassembler ILDASM.

Figure 3 Disassembling vdtor.exe
Figure 3 Disassembling vdtor.exe

Figure 3 shows the disassembled code for vdtor.cpp. Both CBase and CDerived have Finalize methods; Figure 4 shows the Finalize method for the derived class, CDerived. It also shows that the compiler creates a special __dtor method for each class. This method is invoked when you call delete. If you examine the Microsoft® intermediate language (IL) code for the main entry point (main function), you'll see the following lines:

// delete pBase;
IL_0008:  ldloc.0
IL_0009:  call instance void CBase::__dtor()

Figure 4 IL for Finalize Method (Close Figure 4)
.method family virtual instance void  Finalize() cil managed {
  // Code size       18 (0x12)
  .maxstack  1
  IL_0000:  ldsflda    valuetype $ArrayType$0x6e2836de modopt([Microsoft.VisualC]Microsoft.VisualC.IsConstModifier) ??_C@_0BA@PPNICPAI@dtor?3?5CDerived?6?$AA@
  IL_0005:  call       vararg int32 modopt([mscorlib]System.Runtime.CompilerServices.CallConvCdecl) printf(int8 modopt([Microsoft.VisualC]Microsoft.VisualC.NoSignSpecifiedModifier) modopt([Microsoft.VisualC]Microsoft.VisualC.IsConstModifier)*)
  IL_000a:  pop
  IL_000b:  ldarg.0
  IL_000c:  call       instance void CBase::Finalize()
  IL_0011:  ret
} // end of method CDerived::Finalize

When you delete a managed object, the compiler generates a call to the special __dtor method. But which __dtor method does the compiler call? Since I declared pBase as a pointer-to-CBase (CBase*), the compiler calls CBase::__dtor, as the preceding snippet shows. This would seem to imply that CDerived is bypassed during destruction, until you look at the implementation for CBase::__dtor:

// in CBase::__dtor()
IL_0000:  ldarg.0
IL_0001:  call void [mscorlib]System.GC::SuppressFinalize(object)
IL_0006:  ldarg.0
IL_0007:  callvirt   instance void CBase::Finalize()
IL_000c:  ret

The __dtor function calls Finalize using callvirt which, even if you've never heard of IL, you can guess is the instruction to call a method virtually. The common language runtime (CLR) calls the Finalize method for whichever class the object actually is—in this case, CDerived. To enforce the expected C++ destructor semantics, each Finalize method explicitly calls its base class Finalize method, as you can see if you examine CDerived::Finalize:

// in CDerived::Finalize()
IL_000b:  ldarg.0
IL_000c:  call instance void CBase::Finalize()
Here the compiler generates a normal call, not callvirt, instruction. Otherwise your program would recurse until it ran off the end of its stack.

You'll notice that CBase::__dtor calls SuppressFinalize before calling your Finalize method. Why is that? Because in C++, when you delete a managed object, the system doesn't free that object's storage. At least, not immediately. The object's memory isn't freed until the garbage collector runs. SuppressFinalize is required to avoid Finalizing the object twice—first when you call delete and again when the garbage collector runs. Got it?

Q Is there a way to call an MFC extension DLL from a Microsoft .NET Framework assembly? I know how to use P/Invoke to call a regular DLL or to call a COM DLL, but I don't know how to deal with an MFC extension DLL.

Ali Zamurad

A I'm afraid the short answer is: don't go there. In theory, it should be possible to call an MFC extension DLL from managed code; in practice it would be extremely difficult, if not impossible. MFC extension DLLs hook into a lot of MFC state, which is difficult to create from managed code. For example, when you call AfxGetApp to get the application object, this presumes a pointer to your CWinApp has been initialized as a global in MFC's module state. If you've written COM objects in MFC, you know that you need AFX_MANAGE_STATE or METHOD_PROLOGUE macros at the beginning of every entry point to initialize MFC's state.

MFC extension DLLs share a CWinApp-derived global with the main EXE or DLL that hosts the extension. So if you create a new EXE (even a native one), there's no way to load an MFC extension DLL without an application object. The main EXE has to be an MFC app (possibly with some parts compiled using /clr); or you have to rewrite your DLL to not be an extension DLL. If you have back-end code that implements business logic or algorithms, you should be able to isolate it in C extern functions that require no state, which you can call using P/Invoke; or, alternatively, wrap your logic in managed classes (see my September 2005 column at C++ At Work: Copy Constructors, Assignment Operators, and More or the .NET documentation for details). As for user-interface code, the Redmondtonians advise against it. There's just no reasonable way to invoke MFC from managed code. Sorry.

Q I have a class library that's written in C++ and I'm in the process of exposing it to .NET using the Managed Extensions. Some of my functions use uint (unsigned integer), which corresponds to UInt32 in .NET. The other day I was reading the fine print and I noticed that UInt32 is "not CLS-compliant." What exactly does this mean, and should I worry about it?

Dave Layton

A The .NET CLR is the runtime system that loads and executes managed assemblies. The CLR is what lets you write .NET-based applications in any language you want, provided there's an IL compiler for it. But different languages support different features. For example, C++ has unsigned int and C# has uint, but Visual Basic® .NET currently has no equivalent built-in type for unsigned integers (it gets supports for them in Visual Basic 2005). If you want your objects to fully interact with other objects regardless of which language they're implemented in, you must restrict yourself to a smaller subset of the full type system. This subset is defined by the Common Language Specification (CLS), which you can find in the topic "Cross-Language Interoperability" in the .NET Framework documentation. Among many other things, the CLS specifies which built-in types are CLS-compliant.

Figure 5 shows part of the specification that deals with integer types. As you can see, the unsigned types are not CLS-compliant. That's because at the time the Framework was developed, Visual Basic did not have a built-in unsigned type.

Figure 5 Integral Types (Close Figure 5)

CLR Class | Description | CLS-Compliant | Visual Basic | C# | C++
Int16 | A 16-bit signed integer. | Yes | Short | short | short
Int32 | A 32-bit signed integer. | Yes | Integer | int | int , long
Int64 | A 64-bit signed integer. | Yes | Long | long | __int64
UInt16 | A 16-bit unsigned integer. | No | UInt16. No built-in type. | ushort | unsigned short
UInt32 | A 32-bit unsigned integer. | No | UInt32. No built-in type. | uint | unsigned int,unsigned long
UInt64 | A 64-bit unsigned integer. | No | UInt64. No built-in type. | ulong | unsigned __int64
IntPtr | A signed integer whose size depends on the underlying platform (a 32-bit value on a 32-bit platform and a 64-bit value on a 64-bit platform). | Yes | IntPtr. No built-in type. | IntPtr. No built-in type. | IntPtr. No built-in type.
UIntPtr | An unsigned integer whose size depends on the underlying platform (a 32- bit value on a 32-bit platform and a 64-bit value on a 64-bit platform). | No | UIntPtr. No built-in type. | UIntPtr. No built-in type. | UIntPtr. No built-in type.

What does it mean that UInt32 is not CLS-compliant? In particular, what does it mean for your library port? First of all, the CLS rules apply only to the classes and methods your assembly exposes to the outside world. Internally, you can use whatever types your language supports. Second, UInt32 is a full-fledged CLR class, which means any .NET-targeted language, including Visual Basic, can compile and link with your code, even if it exposes methods that use UInt32. If you have a method that returns an UInt32, either as a return value or [out] parameter, any program can then pass that value back to another method as an input parameter. But some languages like Visual Basic may not be able to create an unsigned integer. This may or may not be a problem, depending on your application.

A Visual Basic program can always pass a negative Integer, and when it gets to your library, it'll be treated as unsigned—but the Visual Basic program can't do arithmetic properly, since it treats large unsigned values as negative integers. If you need arithmetic and the full 32-bit range of an unsigned integer, you should expose your parameter as Int64 instead of UInt32.

How do you know if your app is CLS-compliant? If you're writing in C#, you can use the CLSCompliant attribute to have the compiler check your code for CLS compliance. You can apply CLSCompliant to your entire assembly or a particular class or method. For example:

// mark entire assembly as CLS-compliant
[assembly:CLSCompliant(true)];
To mark a particular class or method as CLS-compliant, you'd apply the attribute to the class/method.

Alas, while the C++ compiler recognizes the CLSCompliant attribute, it doesn't check for compliance. That is, the C++ compiler won't complain of noncompliant code, even if you mark it as compliant. There's a separate standalone tool called FxCop that's like lint for managed assemblies (for those of you who remember what lint is), but while FxCop will complain about all sorts of things (such as empty destructors, class names that begin with "C", and variable names that aren't composed of English words) it doesn't check for CLS compliancy, which you would think is more useful. So I'm afraid as of this writing, there's no easy way to automatically check managed C++ programs for CLS compliancy.

Happy programming!

Send your questions and comments for Paul to cppqa@microsoft.com.

Paul DiLascia is a freelance software consultant and Web/UI designer-at-large. He is the author of Windows++: Writing Reusable Windows Code in C++ (Addison-Wesley, 1992). In his spare time, Paul develops PixieLib, an MFC class library available at www.dilascia.com.


{ End Bracket }: Phoenix Rising

{ End Bracket }
Phoenix Rising
Guy Eddon

Phoenix is neither a compiler nor a JITer, but will eventually transform both. It is the codename for an internal Microsoft project that provides an extensible framework for the analysis, optimization, and modification of code during compilation. Think of Phoenix as a tool that allows you to peek into the black box at the heart of compilers in order to see and modify their internals. Current uses of Phoenix inside Microsoft include code obfuscation, visualization, and custom assembly rewriting. Phoenix is the result of a joint endeavor by Microsoft Research and the Developer Division, and is currently available in pre-beta form for noncommercial use to qualified researchers affiliated with recognized institutions (see www.research.microsoft.com/phoenix).

The Phoenix toolkit provides a prototype that will, in time, replace the Microsoft standard compiler back end (c2.exe), as well as the Microsoft® .NET Framework just-in-time (JIT) compiler (mscorpjt.dll) and pre-JITer (ngen.exe). The Phoenix c2 back end can generate code for two distinct target instruction sets: Intel machine code (both 32- and 64-bit) and Microsoft intermediate language (IL). Furthermore, using these Phoenix-enabled compilers, it is easy to build custom compiler plug-ins simply by extending the Phx.PlugIn and Phx.Phase classes and then overriding the Phase.Execute method with your own code.

Indeed, plug-ins have complete access to all the compiler's internal data structures. For example, the following code fragment implements a compiler phase that displays the name of each method processed. (A more sophisticated compiler plug-in might implement an advanced optimization.)

public class DisplayNamesPhase : Phx.Phase {
   protected override void Execute(Phx.Unit unit) {
      Phx.FuncUnit func = unit as Phx.FuncUnit;
      Console.WriteLine("Function name: {0}", func.NameString);
   }
}

Thus, Phoenix makes it straightforward to implement compiler extensions without writing a new compiler from scratch—in effect, it makes it possible to simply utilize the engineering in Phoenix in order to build extensions that focus solely on the problem you're trying to solve.

Phoenix uses an internal intermediate representation (IR) as the basis for code analysis and transformation. For a program to be rewritten, it first needs to be converted to the Phoenix IR by one of the supplied readers (native code, IL, and abstract syntax tree readers are provided with Phoenix, and others can be written for nonstandard code formats). Once the IR has been instrumented, it can be written out to an executable. From this perspective, the back-end compiler provided by Phoenix is really just an application written on top of the Phoenix framework. It uses Phoenix readers to convert input code to IR, and invokes any requested Phoenix plug-ins to transform the code; finally, it writes out the modified code in the desired format (native or IL). Phoenix programs can therefore be written either as compiler plug-ins or as meta-programming tools run post-compilation.

For a real-life example of Phoenix's utility, suppose you want to analyze what fraction of your code users actually execute on a daily basis by building an instrumented version of your program that collects and records such data. Traditionally, you'd need to write a program that reads in a Portable Executable (PE) binary file, figures out what's code and what's data, identifies each function, basic block, and instruction, inserts the instrumentation code, recomputes all the addresses to adjust for the added code, and then writes out a valid PE binary. With Phoenix, by contrast, you just need to write a program that adds the instrumentation at the appropriate places in the binary; all the grunt work of reading and writing the PE file, identifying elements in the binary code, and recomputing addresses is done for you. Due to its extensible nature, Phoenix has been attracting the interest of computer scientists involved in programming language research.

I've recently been using Phoenix to integrate a transaction processing model with object-oriented programming languages like C# and Java. As in databases, the goal is to make a block of statements succeed or fail as an atomic unit. If code in an atomic block throws an exception, the transaction is rolled back and all mutations must be undone. Before Phoenix, I would have had to write a new compiler or modify an existing one in order to test these ideas. With a framework like Phoenix, however, I can simply implement this code transformation as a compiler plug-in and rely on a commercial-quality compiler.

Guy Eddon is a PhD candidate in Computer Science at Brown University where his research focuses on autonomic computing. Prior to graduate school, he worked for over a decade in IT and wrote about component software for Microsoft Systems Journal and Microsoft Press.