ࡱ > S U @ A B C D E F G H I J K L M N O P Q R o@ Cf bjbj p p m o o ѱ d d d d d d d d 8 d ` . ^ | $ R d H " H H d d a 6 6 6 H 2 d d . 6 H 6 6 7 r % d d p T l z* ` w 0 d/ p d d d d d d d p X { 6 i S d d 2 d 5 " d d 2 Chapter 11 Troubleshooting IIS6.0 When administering Web applications, you will have problems to solve whether fixing a service interruption, analyzing a configuration problem, or working around system resource contention problems. Whether the application is an internal corporate Web-presence or a widely available, scaled-out, data-intensive Web application, any problems that you encounter can seem nearly impossible to resolve, and attempting to do so consumes valuable time and resources. This chapter describes how to troubleshoot Web applications on production computers running the Microsoft Windows Server2003 operating system and Internet Information Services (IIS)6.0. It explains how to use troubleshooting tools to administer IIS and discusses in detail service and program areas that might require troubleshooting, such as security, service startup, the IIS metabase, HTTP, and ASP. In This Chapter TOC \h \z \t "Heading 1,1" HYPERLINK \l "_Toc64881753" Overview of Troubleshooting IIS6.0 PAGEREF _Toc64881753 \h 1 HYPERLINK \l "_Toc64881754" Tools for Troubleshooting IIS6.0 PAGEREF _Toc64881754 \h 4 HYPERLINK \l "_Toc64881755" IIS Fundamentals PAGEREF _Toc64881755 \h 12 HYPERLINK \l "_Toc64881756" Common Troubleshooting Tasks PAGEREF _Toc64881756 \h 17 HYPERLINK \l "_Toc64881757" HTTP Status Codes PAGEREF _Toc64881757 \h 26 HYPERLINK \l "_Toc64881758" Troubleshooting Configuration Problems PAGEREF _Toc64881758 \h 42 HYPERLINK \l "_Toc64881759" Troubleshooting HTTP.sys Problems PAGEREF _Toc64881759 \h 52 HYPERLINK \l "_Toc64881760" Troubleshooting Miscellaneous Problems PAGEREF _Toc64881760 \h 53 HYPERLINK \l "_Toc64881761" Advanced Troubleshooting PAGEREF _Toc64881761 \h 56 HYPERLINK \l "_Toc64881762" Additional Resources PAGEREF _Toc64881762 \h 60 Related Information For more information about troubleshooting, see Troubleshooting in IIS6.0 Help, which is accessible from IISManager. Overview of Troubleshooting IIS6.0 Problems that occur when administering Internet Information Services (IIS)6.0 might be difficult to solve for any of the following reasons: Problem reports from end users might be vague or incomplete, lacking such information as the content of error messages, how frequently a problem occurs, or the specific conditions under which the problem occurred. Logs do not record information that reveals the cause of the problem. The problem occurs in a complex, widely distributed application, which is difficult to track comprehensively. The goal of troubleshooting is to isolate and determine the causes of problems that interfere with normal operation. This section describes a troubleshooting methodology and some of the tools that you can use to troubleshoot a server running IIS6.0 and the Web applications that it hosts. Troubleshooting Methodology To efficiently troubleshoot Web applications, you need to apply and consistently use a troubleshooting framework, or methodology. Doing so helps you to streamline your troubleshooting activities and approach problems with confidence. The fundamental troubleshooting methodology has five phases: Phase 1: Discovery. Gather information about the problem. Phase 2: Planning. Create a plan of action. Phase 3: Problem Reproduction. Reproduce the problem, or determine that you cannot reproduce it. If you cannot reproduce the problem, then you might not have enough information to confirm that there is a problem. Phase 4: Problem Isolation. Isolate the variables that relate directly to the problem. Phase 5: Analysis. Analyze your findings to determine the cause of the problem. Phase 1: Discovery Most problems are discovered by the end user. Problems might also be revealed by the operating system or by application logging and event tracking utilities. When a problem is reported, begin the troubleshooting process by gathering information to confirm that a problem exists. Interview the user who reported the problem If an end user reported the problem, ask that user for detailed information about what occurred. If possible, get the exact text of any error messages. If the problem recurs, ask the user to provide screen shots for error-message information or other symptoms that appeared on a screen. If the problem is behavioral and does not generate an error message, ask very specific, closed-ended questions. For example, you might ask the following questions: What time did the problem occur? (You can use this information to locate proximate events a large log file or to correlate the problem with external causes.) Which button(s) or link(s) did you click just before the problem occurred? Did you refresh or re-request the Web page to recover from the problem? Examine the records The Microsoft Windows Server2003, Standard Edition; Windows Server2003, Enterprise Edition; Windows Server2003, Web Edition; and Windows Server2003, Datacenter Edition operating systems provide logging and event tracking. Check the logs for entries around the time that the problem occurred. Additionally, the application itself might log events. For example, the IIS logs and the HTTP error log record this information. For more information, see Analyzing Log Filesiisrg_log_GADT in this book. Phase 2: Planning Creating a plan of action is quite possibly the most important phase in the troubleshooting process. In this phase, you list the steps that you will follow as you proceed through the other troubleshooting phases. Stick to this plan and refer to it often. When you are bogged down in the tasks of the problem isolation phase, referring to your plan will help you remember where you are in the troubleshooting process and avoid getting sidetracked. Revise the plan as you progress in most cases one step will dictate later steps. When troubleshooting IISrelated problems, your plan might consist of configuring IIS logging to log extra details and then setting up a Performance Monitor log to run during a specific period of time. Based on the results of these actions, the next step might involve making a configuration change. Phase 3: Problem Reproduction Determine whether the problem is readily reproducible. The ability to reproduce the problem on demand is fundamental to properly and efficiently troubleshooting the problem. Use the information that you gathered in Phase 1 to reproduce the problem, most often by repeating the actions taken by the end user before the problem occurred. If you can readily reproduce the problem, the isolation phase is more manageable. When you have determined a set of steps or events that trigger the problem, move to the isolation phase. If you cannot readily reproduce the problem, the problem isolation phase can be tedious or isolating the problem can be impossible. If you cannot reproduce the problem, prepare to gather the right kind of information the next time it happens. Consider doing any or all of the following: Enable more detailed event tracking. Ask users to watch for the problem and pay close attention to what they are doing if it occurs. Write additional code in the application that looks for and highlights the problem if it happens again. For example, when dealing with a Web application problem, you might add extra tracing code to your ASP pages or you might configure Windows security auditing for specific failures. The goal is to gather enough information so that the next time the problem occurs, the information that you obtain will be sufficient to correctly diagnose the problem. Phase 4: Problem Isolation In the problem isolation phase, you reproduce the problem as efficiently as possible by using repetitive steps. In this phase, you eliminate variables such as settings, file actions, component starts/stops, or any change in the execution that do not cause the problem, narrowing the variables down to those that are responsible for the problem. You have succeeded in isolating the problem when you achieve the following conditions: The problem can be reproduced consistently when you take a fixed series of actions. The problem cannot be reproduced when you omit any of those actions. For example, when troubleshooting IISrelated problems, you might do the following: Configure a Web application to be hosted in its own worker process or configure a component to execute in an isolated host process like a COM+ Dllhost.exe in order to determine which part of an application is consuming CPU time or leaking memory. Configure security settings on a URL in varying degrees through repetitive tests to isolate authentication problems. Capture performance data to determine whether a problem lies in the core Web server or in a Web service extensionbased application, such as ASP or ASP.NET. Phase 5: Analysis Depending on the nature of the problem, the analysis phase can be the most difficult phase of the troubleshooting process. In the analysis phase, you use everything that you learned in the previous phases to do the following: Determine the cause of the problem. Explore how the problem affects the application. Determine the best way to solve the problem. This step might be a simple formality if, after Phase 3 is complete, you know what the problem is and how to fix it. If not, you must perform additional analysis of the data you have gathered. You might find that you must spend more time in the isolation phase. You might find that there are several ways to fix the problem. Take what you have learned about the problem and decide which actions to take. Ask yourself these kinds of questions: What is the impact of the problem? Do the benefits of fixing the problem outweigh the costs of fixing it? Is there an acceptable workaround? Record your progress and the data that you have gathered. As you obtain information about a problem, you will make decisions about the steps that you need to take that will often alter your plan of action. It is important to record the information that you collect so that you have a way to account for your decisions as you move through the process of troubleshooting. If you find yourself taking the wrong path, having this information will allow you to backtrack to the point where you made the wrong decision. In addition, your ability to explain the changes or improvements in the problem is often important to other stakeholders. Tools for Troubleshooting IIS6.0 Good tools are an indispensable part of any troubleshooting effort. This section identifies tools that you can use to help troubleshoot problems that you encounter on applications running on IIS6.0. It also provides tips for getting the most benefit from each tool. Specific tool usage scenarios are discussed later in this chapter. The following tools are useful for troubleshooting problems that occur with IIS6.0: WFetch (WFetch.exe). This tool allows you to generate customized HTTP requests and view request and response data. File Monitor (FileMon.exe). This tool allows you to view and capture real-time file system activity. Registry Monitor (RegMon.exe). This tool allows you to view and capture real-time system registry activity. IIS Enterprise Tracing for Windows. This tool allows you to enable tracing for various providers. IIS includes a set of providers that can be used to troubleshoot various internal processes in addition to application performance. IIS Enterprise Tracing for Windows is new in WindowsServer2003. Logman.exe and Tracerpt.exe are the tools used to enable IIS ETW tracing. Network Monitor. This network tracing utility allows you to view activity in the network stack. Network Monitor is included with Windows Server2003. System Monitor. This tool allows you to view and collect system performance data. System Monitor is included with Windows Server2003. HRPlus. This tool provides error lookup functionality. Microsoft Debugging Tools for Windows. This tool allows you to debug and diagnose application problems. WFetch A simple HTTP request that is issued by typing a URL in the address bar of a browser might produce multiple network requests to the target server running IIS. However, the user does not see all of the resulting response data because the browser does not display it. WFetch allows you to fully customize an HTTP request and send it to a Web server so that you can see the raw HTTP request and response data. WFetch is included in the Internet Information Services (IIS)6.0 Resource Kit. For more information about using WFetch in a testing environment, see article 284285, How To: Use Wfetch.exe to Troubleshoot HTTP Connections in the Microsoft Knowledge Base. To find this article, see the Microsoft Knowledge Base link on the Web Resources page at http://www.microsoft.com/windows/reskits/webresources. File Monitor and Registry Monitor When troubleshooting IIS6.0, you can use File Monitor and Registry Monitor to obtain two specific types of information: Verification that a specific file (File Monitor) or registry key (Registry Monitor) is being accessed by a given process. Information about whether that access results in an Access Denied error. Both File Monitor and Registry Monitor can be downloaded from the System Internals Web site. To install File Monitor and Registry Monitor, unzip and copy each .exe file to a desired location. To use one of these tools, double-click the appropriate executable file (FileMon.exe or RegMon.exe). Both utilities start in capture mode, which means that they display all file system (File Monitor) or registry (Registry Monitor) activity from all processes as it happens. Before you begin troubleshooting a problem, stop the capture and clear the entries that have been written. To stop or start a capture in FileMon.exe or RegMon.exe On the Options menu, click Capture Events. To clear the current capture in FileMon.exe or RegMon.exe On the Edit menu, click Clear Display. Tips for Using File Monitor and Registry Monitor for Troubleshooting Use the following tips to streamline the troubleshooting process when using File Monitor and Registry Monitor: Isolate your task as much as possible. The output from a capture comes very quickly and at a high volume. The larger the capture, the harder it is to find the information that you want. When you are ready, perform only the tasks that will produce the file or registry access that you are trying to verify. Avoid unnecessary clicking or activity. Performing extraneous operations while File Monitor or Registry Monitor is running will produce unrelated and unnecessary data in the output. Start a capture as closely as possible to when you perform the task. Separate your utility window from the workspace that contains the process that you are using to troubleshoot so that you can easily switch between programs. If possible, press ALT+TAB to quickly switch programs. Create a capture filter for the process name that you are working with. Registry Monitor and File Monitor both have capture filter functionality, which you can use to remove extraneous activity from the capture. You can filter on any string in an entry. For example, a common filter is the IIS6.0 worker process name, W3wp.exe. To set a capture filter for W3wp.exe On the Edit menu of the appropriate tool, click Filter/Highlight. In the Include box, type W3wp.exe and then click Apply. When you have finished the task, stop the capture and scan the results for specific output. Usually, your objective will be to verify whether the problem is being caused by or is related to an access denial. In either tool, you can see whether this is the case by looking for the text ACCESS DENIED in the Result column. If you need to compare the time that an event occurred with some other activity on your network, change the time format of the capture from the elapsed time, which is the default, to the literal time of day. IIS6.0 Enterprise Tracing for Windows WindowsServer2003 includes Enterprise Tracing for Windows (ETW), a new framework for implementing tracing providers that can be used for debugging and capacity planning. IIS6.0 implements a number of providers to trace key transitions and the progress that an HTTP request makes as it moves through the IIS service framework. These providers include the following: HTTP Service Trace. This provider traces new connections, new HTTP requests from a given connection, HTTP kernel cache hits, application pool routing, and HTTP request completion. IIS: WWW Server. This provider traces the transition of a request from HTTP.sys to the worker process, start and stop events surrounding Common Gateway Interface (CGI) requests, and the transition of a request back to HTTP.sys from the worker process. This information reveals the type of request that is being processed. IIS: SSL Filter. This provider traces calls into and out of the SSL Filter, which is used for SSL traffic. IIS ISAPI Extension. This provider traces the transition of a request into and out of ISAPI extension processing. IIS ASP. Traces the start and completion of the execution of an ASP request. These tracing providers allow you to follow a request through all of its phases. This information is very useful in the problem isolation phase of troubleshooting. For example, ISAPI filters can often cause delays or hangs in HTTP request processing. The tracing functionality can help to determine which ISAPI filter is causing the delay for a given request. Use the LogMan utility, which is included with WindowsServer2003, to configure and control ETW. All of the tracing providers are enabled through their Globally Unique ID (GUID), which you can find by using LogMan. A GUID is a 128-bit number that is generally represented by a 32character string of hexadecimal numbers. The following procedure demonstrates one way that ETW tracing can be used. To capture an ETW tracing session List the GUIDs of the providers by using the LogMan utility. At a command prompt, type the following: logman.exe query providers The output will look similar to the following: D:\>logman.exe query providers Provider GUID ------------------------------------------------------------------------------- ACPI Driver Trace Provider {dab01d4d-2d48-477d-b1c3-daad0ce6f06b} Active Directory: Kerberos {bba3add2-c229-4cdb-ae2b-57eb6966b0c4} IIS: SSL Filter {1fbecc45-c060-4e7c-8a0e-0dbd6116181b} IIS: WWW Server {3a2a4e84-4c21-4981-ae10-3fda0d9b0f83} IIS: Active Server Pages (ASP) {06b94d9a-b15e-456e-a4ef-37c984a2cb4b} Local Security Authority (LSA) {cc85922f-db41-11d2-9244-006008269001} Windows Kernel Trace {9e814aad-3204-11d2-9a82-006008a86939} ASP.NET Events {AFF081FE-0247-4275-9C4E-021F3DC1DA35} NTLM Security Protocol {C92CF544-91B3-4dc0-8E11-C580339A0BF8} IIS: WWW ISAPI Extension {a1c2040e-8840-4c31-ba11-9871031a19ea} HTTP Service Trace {dd5ef90a-6398-47a4-ad34-4dcecdef795f} Active Directory: NetLogon {f33959b4-dbec-11d2-895b-00c04f79ab69} Spooler Trace Control {94a984ef-f525-4bf1-be3c-ef374056a592} The command completed successfully. D:\> Copy the GUIDs of the providers that you want to use into a text file, specify flags and levels, and then save the file using an appropriate name, such as iis_providers.guid. For information about setting flags and levels, see Create and Configure Trace Logs in Help and Support Center for Windows Server2003. The file should look similar to the following, where the # character delimits comments: #GUID#flags #level #provider {1fbecc45-c060-4e7c-8a0e-0dbd6116181b}0 5#IIS: SSL Filter {3a2a4e84-4c21-4981-ae10-3fda0d9b0f83}0 5#IIS: WWW Server {06b94d9a-b15e-456e-a4ef-37c984a2cb4b}0 5#IIS: Active Server Pages (ASP) {AFF081FE-0247-4275-9C4E-021F3DC1DA35}0 5#ASP.NET Events {a1c2040e-8840-4c31-ba11-9871031a19ea}0 5#IIS: WWW ISAPI Extension {dd5ef90a-6398-47a4-ad34-4dcecdef795f}0 5#HTTP Service Trace Start the tracing session. At the command prompt, type the following: logman.exe start iis_trace -pf iis_providers.guid -ct perf -o iis_output.etl -bs 64 -nb 200 400 -ets The output should look similar to the following: D:\>logman start iis_trace -pf iis_providers.guid -ct perf -o iis_output.etl -bs 64 -nb 200 400 -ets Name: iis_trace Age Limit: 15 Buffer Size: 64 Buffers Written: 1 Clock Type: Perf Events Lost: 0 Flush Timer: 0 Buffers Free: 159 Buffers Lost: 0 File Mode: Sequential File Name: D:\iis_output.etl Logger Id: 3 Logger Thread Id: 3732 Maximum Buffers: 160 Maximum File Size: 0 Minimum Buffers: 160 Number of buffers: 160 Real Time Buffers Lost: 0 Provider Flags Level ------------------------------------------------------------------------------- * "HTTP Service Trace" 0x00000000 0x05 {DD5EF90A-6398-47A4-AD34-4DCECDEF795F} 0x00000000 0x05 * "IIS: SSL Filter" 0x00000000 0x05 {1FBECC45-C060-4E7C-8A0E-0DBD6116181B} 0x00000000 0x05 * "IIS: WWW Server" 0x00000000 0x05 {3A2A4E84-4C21-4981-AE10-3FDA0D9B0F83} 0x00000000 0x05 * "IIS: WWW ISAPI Extension" 0x00000000 0x05 {A1C2040E-8840-4C31-BA11-9871031A19EA} 0x00000000 0x05 * "IIS: Active Server Pages (ASP)" 0x00000000 0x05 {06B94D9A-B15E-456E-A4EF-37C984A2CB4B} 0x00000000 0x05 The command completed successfully. D:\> Attempt to reproduce the problem that you are troubleshooting. After you have reproduced the problem, stop the tracing session. At a command prompt, type the following: logman.exe stop iis_trace ets Note When tracing is enabled, it is enabled for all requests. If you need trace data for only the request that is having problems, you should disconnect the server from the network before running the test. Tracing generates a binary file. To obtain readable information, use a report generation tool like TraceRpt.exe. To generate an ETW tracing report Using the preceding output file as an example, at a command prompt, type the following: tracerpt.exe iis_output.etl -o iis_logdump.csv -summary iis_trace_summary.txt -report iis_trace_report.txt This command generates the specified reports, as follows: D:\>tracerpt iis_output.etl -o iis_logdump.csv -summary iis_trace_summary.txt -r export iis_trace_report.txt Input ---------------- File(s): iis_output.etl 100.00% Output ---------------- Text (CSV): iis_logdump.csv Summary: iis_trace_summary.txt Report: iis_trace_report.txt The command completed successfully. You can examine the .txt files for high-level information that was captured in the trace. The .csv file can be opened in Microsoft Exc e l o r i m p o r t e d t o a d a t a b a s e , s u c h a s M i c r o s o f t S Q L S e r v e r "!. N e t w o r k M o n i t o r W h e n t r o u b l e s h o o t i n g I I S r e l a t e d p r o b l e m s , y o u c a n u s e N e t w o r k M o n i t o r t o s t a r t a c a p t u r e , r e p r o d u c e a p r o b l e m , a n d v i e w t h e c a p t u r e . N e t w o r k M o n i t o r i s i n c l u d e d w i t h W i n d o w s S e r ver2003. To install Network Monitor From the Start menu, click Control Panel. Double-click Add or Remove Programs. In the left pane, click Add/Remove Windows Components. Click Management and Monitoring Tools and then click Details. Select the Network Monitor Tools check box and then click OK. Click Next. When installation is complete, click Finish. Network Monitor is now available on the Administrative Tools menu. Note Network Monitor, unlike WFetch, is a good tool for analyzing TCP connection management or three-way handshake information. For more information about using Network Monitor in this way, see article 172983, Explanation of the Three-Way Handshake Via TCP/IP in the Microsoft Knowledge Base. To find this article, see the Microsoft Knowledge Base link on the Web Resources page at http://www.microsoft.com/windows/reskits/webresources. Tips for Using Network Monitor for Troubleshooting Consider the following tips when using Network Monitor for troubleshooting: As with any utility that captures large volumes of data, it is important to plan your Network Monitor capture and to time the beginning and end of captures so that they minimize extraneous network traffic. Network Monitor, like File Monitor and Registry Monitor, can display the timestamp instead of the elapsed time in the capture. Displaying the timestamp along with synchronizing system clocks on participating computers can help you accurately isolate problems. Increase the capture buffer size from its default of 1MB to 20MB. To adjust Network Monitor capture settings On the Capture menu, click Buffer Settings. In the Buffer Size (MB) box, type 20. Click OK. Applying filters to captures allows you to record only relevant data. You can also apply a filter after the capture has completed to filter the data that is displayed. Filtering by protocol and by network addresses are the two most common ways to quickly filter Network Monitor capture data. To filter capture data by HTTP protocol While viewing a saved capture file, on the Display menu, click Filter. In the Display Filter dialog box, click Protocol == Any, and then click Edit Expression. In the Expression dialog box, click Disable All. In the Disabled Protocols list box, click HTTP, and then click Enable. Click OK twice. Network Monitor filters and then displays only HTTP traffic. To filter capture data by network addresses While viewing a saved capture file, on the Display menu, click Filter. In the Display Filter dialog box, click ANY<-->ANY, and then click Edit Expression. In the Expression dialog box, choose the source and destination network. If you dont see the one you want, type the address that you want in the Edit Addresses box. By default, Network Monitor only parses traffic on port 80 as HTTP traffic. You can customize Network Monitor to parse traffic on other ports, for example, on port 8080, as HTTP traffic. To configure Network Monitor to parse traffic on non-standard HTTP ports Open Notepad. Open the Tcpip.ini file, which is located in the Parsers folder under the Network Monitor installation point (by default, systemroot\System32\Netmon). Locate the [TCP_HandoffSet] section. Below the 80=HTTP line, type the following new line: 8080=HTTP Save and then close the Tcpip.ini file. Open the Http.ini file, which is located in the same folder as Tcpip.ini. Locate the [Ports] section. Immediately after the text SrcPorts=80, type the new port, preceded by a comma. The line now reads SrcPorts=80,8080. Save and then close the Http.ini file. To start any new parsing activity, close and reopen your captures. For more information about HTTP traffic and Network Monitor, see article 252876, How to View HTTP Data Frames Using Network Monitor in the Microsoft Knowledge Base. For more information about capturing network traffic, see article 148942, How to Capture Network Traffic with Network Monitor in the Microsoft Knowledge Base. To find these articles, see the Microsoft Knowledge Base link on the Web Resources page at http://www.microsoft.com/windows/reskits/webresources. System Monitor System Monitor is part of the Performance snap-in, which is accessible through Administrative Tools. When generating a performance log for troubleshooting, add, at a minimum, the following counter objects: .NET CLR (Common Language Runtime) Active Server Pages (ASP) ASP.NET Memory Process Processor Thread Web Service Alternatively, you can observe real-time data on the display, if doing so meets your troubleshooting needs. To start System Monitor From the Start menu, point to Administrative Tools, and click Performance. Note If System Monitor does not open by default, on the console tree, click System Monitor. HRPlus HRESULTS are a type of 32-bit error code returned from various parts of the Windows Server2003 operating system and its components. HRPlus is a free tool that is used primarily for decoding HRESULT values returned by COM applications, but it also can be used to generate custom HRESULT values for use in your own applications. Additionally, you can use HRPlus to decode Win32 error codes. To obtain a copy of HRPlus, see the CNET Shareware.com link on the Web Resources page at http://www.microsoft.com/windows/reskits/webresources, and then search for hrplus. Microsoft Debugging Tools for Windows The Microsoft Debugging Tools for Windows toolkit provides a robust suite of utilities for troubleshooting and debugging software. To download Microsoft Debugging Tools for Windows, see the Debugging Tools link on the Web Resources page at http://www.microsoft.com/windows/reskits/webresources. For more information about Microsoft Debugging Tools for Windows, see Advanced Troubleshootingiisrg_trb_KMFN later in this chapter. IIS Fundamentals IIS is an HTTP protocol server, which means that it accepts and replies to HTTP requests. IIS processes three types of HTTP requests: Static content (like HTM document) requests CGI requests ISAPI requests By understanding the fundamentals of IIS architecture and the related functionality, you can discover and isolate problems much more quickly. You will also be better equipped to differentiate between real problems and expected behavior. This section explores the following functionality, which any IIS troubleshooter must understand: HTTP protocol basics. Explains the way Internet browsers and servers running IIS communicate with each other. IIS service startup. Sometimes problems arise even before IIS starts servicing requests. This section describes the startup process and typical startup behavior, which can help you determine the causes of problems. HTTP request walkthrough. Explains the various phases that a typical request passes through as it moves through the service. HTTP Protocol Basics Troubleshooting IIS often requires that you examine the communications between a Web client and the server running IIS. By understanding the fundamentals of this communication, you can easily interpret network monitor traces and similar records. Hypertext Transfer Protocol (HTTP) is the communication protocol used to exchange information between a client system and a Web server across a TCP/IP connection. The interchange is generally referred to as an HTTP transaction. In an HTTP transaction, the client system opens a TCP connection with the Web server and submits an HTTP request. The Web server, in turn, issues an HTTP response, completing one HTTP transaction. HTTP is stateless, in that there is no provision in the protocols design for information about any single request persisting beyond one transaction. The following code segment is an example of an HTTP GET request. GET /default.htm HTTP/1.1\r\n Host: rhynoruv\r\n Accept: */*\r\n \r\n The request line contains an HTTP method, a Uniform Resource Identifier (URI), and the HTTP version number, followed by carriage return and line feed characters. Typically, the request line is followed by a series of HTTP headers. Depending on the type of request, an HTTP request can have a variety of headers. With HTTP 1.1, the simplest GET request consists of the request line and only one HTTP h e a d e r t h e H o s t h e a d e r . T w o c a r r i a g e r e t u r n s / l i n e f e e d s d e s i g n a t e t h e e n d o f t h e H T T P r e q u e s t h e a d e r . I n a s i m p l e G E T r e q u e s t , t h e s e i n d i c a t e t h e e n d o f t h e r e q u e s t . S o m e r e q u e s t s i n c l u d e d a t a , w h i c h i s c a l l e d t h e e n t i t y b o d y , i n a d d i t i o n t o t h e H T T P h e a ders. In the HTTP request, the division between the headers and the entity body is designated by two carriage returns/line feeds. The following code sample shows an HTTP response, which is similar to a request: HTTP/1.1 200 OK\r\n Content-Length: 50\r\n Content-Type: text/html\r\n Last-Modified: Sun, 20 Oct 2002 22:52:16 GMT\r\n Accept-Ranges: bytes\r\n ETag: "255591568b78c21:5cd"\r\n Server: Microsoft-IIS/6.0\r\n MicrosoftOfficeWebServer: 5.0_Pub\r\n X-Powered-By: ASP.NET\r\n Date: Sun, 20 Oct 2002 22:52:35 GMT\r\n \r\n \r\n
\r\n \r\n Hello World.\r\n \r\n \r\n The first line is the status line, which consisting of the HTTP version, a numeric HTTP status code, and then a reason phrase that briefly explains the status code. The status code is often the first indicator of a problem. For information about status codes, see HTTP Status Codesiisrg_trb_YBCG later in this chapter. There are two other important aspects of the HTTP protocol that you must understand before you attempt troubleshooting: the provisions for managing TCP connections and how credential management (authentication) occurs. HTTP Connection Management Because the TCP/IP protocol is so robust, establishing a new TCP connection for every HTTP request is relatively expensive in terms of network roundtrips. The two versions of HTTP (1.0 and 1.1) avoid incurring this overhead differently. HTTP 1.0 transactions typically involved a client opening a TCP connection with the Web server and issuing an HTTP request. The server would then send its response and close the TCP connection. The client could optionally ask that the TCP connection remain open beyond the first transaction with the HTTP header Connection: keep-alive. Starting with HTTP 1.1, persistent connections a r e i m p l i c i t ; i n o t h e r w o r d s , t h e W e b c l i e n t a n d W e b s e r v e r k e e p t h e u n d e r l y i n g T C P c o n n e c t i o n o p e n u n t i l e i t h e r s e n d s a n i n d i c a t i o n t h a t t h e c o n n e c t i o n c a n b e c l o s e d w i t h t h e C o n n e c t i o n : c l o s e h e a d e r . B e c a u s e t h e c o n n e c t i o n r e m a i n s o p e n , a w a y t o d e s i g n a t e the end of the entity-body is required. The Content-Length header performs this function by specifying the size of the entity-body, in bytes. HTTP Authentication HTTP contains provisions for negotiating access to a secured resource, a process called authentication. Typically, an initial request from a client will not contain any authentication information; such a request is considered an anonymous request. If the requested resource is secured, the Web server can respond with a 401-Access Denied error and, optionally, can include one or more WWW-Authenticate HTTP headers. Each WWW-Authenticate header indicates a type of authentication scheme the server uses. The client can then choose to engage in one of these authentication schemes; doing so sometimes requires several more HTTP roundtrips. For more information about authentication schemes, see 401.1 and 401.2-Authentication Problemsiisrg_trb_ZXSA later in this chapter. For more information about IIS authentication types, see Managing a Secure IIS6.0 Solutioniisrg_sec_OVERVIEW in this book. Internet RFC 2616, Hypertext Transfer Protocol HTTP/1.1 is the governing specification for the current version of HTTP (1.1). For more detailed information about HTTP, see the Request for Comments (RFC) link on the Web Resources page at http://www.microsoft.com/windows/reskits/webresources. IIS Service Startup On rare occasions, you might encounter problems related to starting the IIS services infrastructure. The IIS services are generally robust enough to handle problems and recover from them gracefully. If a problem does occur and if it is appropriate, an event is logged to the application/system event log explaining the failure. Typically, the startup process never needs troubleshooting beyond examining messages written to the event log and taking the required corrective action. However, because it is helpful to understand the inner workings of an application in order to recognize problematic behavior, this section describes the major stages of the IIS services infrastructure startup process. For information about the components of the services infrastructure, see IIS6.0 Architectureiisrg_arc_OVERVIEW in this book. The Windows Service Control Manager (SCM) performs Windows operating system service management operations. Among other things, SCM is responsible for signaling services to begin the startup process and for managing service dependencies. Some services are configured to auto-start, which means that SCM asks those services to start when the system is restarted. Other services are configured to manually start (or demand-start), which means that SCM starts those services only when prompted by an external request or when a service dependency demands it. IIS services startup begins when the computer restarts. The IIS Admin service, which is hosted by Inetinfo.exe, is configured as an auto-start service. This is the first service in the IIS service infrastructure that is asked by SCM to start. The IIS Admin service depends upon the Remote Procedure Call (RPCSS) service and the Security Accounts Manager (SAMSS) service, which means that SCM guarantees that these are started before the IIS Admin service. The next auto-start service in the IIS services infrastructure that starts is the World Wide Web Publishing Service (WWW service). It is hosted by Svchost.exe. In addition to the IIS Admin and RPCSS services, the WWW service has dependencies on the HTTP SSL (HTTPFilter) service. The IIS Admin service and RPCSS were started earlier, but before SCM starts the WWW service, it needs to demand-start the HTTPFilter service. The HTTPFilter service has dependencies on the IIS Admin service and the HTTP service. It is this dependency on HTTP that causes SCM to start the HTTP.sys kernel-mode driver. Consequently, SCM starts the HTTPFilter service, which runs in Lsass.exe (or Inetinfo.exe if you are running in IIS5.0 isolation mode), and then starts the WWW service. The following sections describe what happens when these services start. HTTP.sys In IIS6.0, unlike in earlier versions of IIS, the HTTP.sys kernel-mode device driver is not part of the core IIS services. In Windows Server2003, the HTTP.sys driver actually exposes an API set that any application or service, not just IIS, can leverage. However, since IIS6.0 is the major client, so to speak, of HTTP.sys, it is beneficial to understand how HTTP.sys relates to IIS6.0. The primary responsibility of HTTP.sys is to listen for and respond to HTTP requests on the network endpoints on which it is requested to do so. When HTTP.sys starts, by default it listens for HTTP requests on all IP addresses configured on the system. In some cases, though, it is beneficial for HTTP.sys to listen only on specific addresses for example, if Windows Media Services needs to listen on port 80 of a specific IP address, outside of the HTTP.sys driver. To accommodate this need, HTTP.sys has an IP inclusion list. This list, when it contains entries, tells HTTP.sys which IP addresses to listen on, and, thereby, which IP addresses not to listen on. At startup, HTTP.sys examines this list and configures itself accordingly. As a result, changes to this list take effect only when the service is restarted. Tip You can use the HttpCfg.exe utility to configure the IP inclusion list. For information about HttpCfg.exe, in Help and Support Center for WindowsServer2003, click Tools, and then click Windows Support Tools. IIS Admin Service The IIS Admin service primarily is responsible for managing the IIS metabase. During startup, it reads the MetaBase.xml file and creates its in-memory representation. Because the IIS Admin service creates this in-memory snapshot, if the metabase file is corrupt or if the XML code in the file is not syntactically well-formed, the IIS Admin service will fail to start. The IIS Admin service is responsible for synchronizing access to the metabase through the various control interfaces, even when manual updates are made directly to the MetaBase.xml file. During startup, the IIS Admin service sets up the control interfaces, like IMSAdmin, and registers for necessary file change notifications from the Windows Server2003 operating system in order to monitor changes to the XML file. WWW Service The WWW service recognizes the following entities: URLs. The combined application and site binding information. Applications. Unique URLs that, when merged with a sites bindings and endpoints, provide full mapping information that enables HTTP.sys to determine which application pool requests that application should be directed to, and the metadata that should be used to support them. Sites. The specific bindings and endpoints that HTTP.sys should listen to. Sites contain one or more applications. Application pools. Sets of applications that are routed to the same set of worker processes. Application pools are also a common configuration for the worker processes that serve the applications in the pool. Application root. Also known as the Web application starting point, an application root is designated when you create and configure an application on a node in a Web site directory structure. You can have more than one application per Web site. The root of a newly created Web site, including the default Web site that is created when you install IIS, is an application starting point. Every file and directory beneath the starting-point directory in your Web site is considered part of the application until another starting-point directory is found. Thus, directory boundaries define the scope of an application. When the WWW service starts, it performs the following operations: Reads WWW Service configuration information from the IIS Admin service. This information includes global data, application pool data, Web site data, and application data. Validates this data and logs any errors to the event log, choosing to ignore a record or to alter properties in the record to the defaults. Configures HTTP.sys for all sites, applications, and application pools that are registered, and starts the ones that are set to start automatically. Enables HTTP.sys configuration all at once, allowing IIS to accept requests. If IIS5.0 isolation mode is enabled, starts a worker process in Inetinfo.exe and waits for the worker process to start. At this point, IIS can accept incoming HTTP requests. The first HTTP request to an application pool launches the worker process (W3wp.exe). In other words, an application pool can be started even though a worker process is not running. Worker Process Startup The fundamental aspects of a worker process startup routine are fairly straightforward. The worker process typically does the following: Starts the metabase listener and listens for change notifications. Initializes the IIS thread pool. Obtains the application pool, which was specified in the command-line parameters, and then sets up the communication mechanism that it will use to receive requests from HTTP.sys. Sets up communication with the WWW service to enable pinging and recycling. Initializes other internal caches and structures that are used to process HTTP requests. Initializes the Web service extension restriction list. Begins accepting requests from HTTP.sys. HTTP Request Walkthrough An HTTP request takes only one of three basic paths through the WWW service depending on whether the request is for a static file, an ISAPI extension, or for CGI processing. This section describes these paths. Understanding the request flow is invaluable during the problem isolation phase of the troubleshooting process. Common Request Operations All HTTP requests arrive from the network and are passed up to the HTTP.sys listener from the TCP protocol stack. HTTP.sys parses the HTTP headers and makes sure that the request is well formed and valid. In most cases, HTTP.sys then determines into which application request queue to place the request. Worker processes are signaled by the presence of requests in the HTTP request queue to start processing the requests. Using the HTTP headers, the worker process determines first which kind of authentication needs to be performed. It is here that the HTTP request goes through the process of getting a user context. All HTTP requests are authenticated (except in the case of anonymous requests served from the HTTP.sys kernel-mode cache) and have a Windows security context, which becomes the user context of the request. Even a request for content that has anonymous access enabled acquires the security context of the configured anonymous account; it is not anonymous in that there is no Windows user whatsoever. Through the authentication process, the worker process obtains a security token for the user by means of a Windows logon API. The user context and the associated Windows security token are fundamental and have a wide-reaching effect on how the rest of the request proceeds. Failures in this area and in areas related to security typically manifest themselves as HTTP 401 errors. For information about troubleshooting this type of error, see HTTP Status Codesiisrg_trb_YBCG later in this chapter. After a user context has been obtained, the worker process determines from the HTTP headers the kind of content the request is requesting, and then chooses the appropriate handler. At this point, the three request types become relevant. Requests for Static Content A request for static content is the most basic kind of request that IIS handles. For this request type, IIS validates that the user has access to the file and then returns the file. This request type also involves validation against the Multipurpose Internet Mail Exchange (MIME) type restriction feature. If the request is for a restricted file type, IIS returns a 404.3-MIME Map Policy Prevents This Request error. For more information, see Requests for Static Files Return 404 Error in IIS6.0 Help. IIS can also cause a response to be cached in the HTTP.sys kernel-mode cache. For more information, see the HTTP.sys Response Cache section in Web Server Scalabilityiisrg_sca_OVERVIEW in this book. Requests Handled by ISAPI Extensions Active Server Pages (ASP) and ASP.NET requests are handled by ISAPI extensions. A request is designated an ISAPI request when it requests a file whose extension is mapped to an ISAPI extension DLL in the ScriptMaps metabase property. A request is also designated an ISAPI request when it requests an ISAPI DLL directly in the URL or when a wildcard ScriptMaps entry exists, whereby all requests go through the mapped ISAPI DLL. IIS calls a well-known entry point in the ISAPI DLL and passes a structure containing all of the information required about a request. When IIS is configured for IIS5.0 isolation mode, ISAPI requests also can be passed out-of-process to a Dllhost.exe worker process. Before an ISAPI request is executed, IIS checks the WebSvcExtRestrictionList metabase property to ensure that the ISAPI is allowed. If the ISAPI is not allowed, IIS returns a 404.2 error. In this kind of request, the ISAPI extension DLL is responsible for processing the HTTP request and then sending the response. When an ISAPI extension DLL processes a request, any number of things can happen. In the case of ASP and ASP.NET, the user context obtained during authentication is typically impersonated before the ScriptMap file is opened. These kinds of requests, where distributed and dynamic operations are performed, often involve custom-built components and backend data stores. CGI Requests CGI requests directly request a CGI executable. They are similar to ISAPI requests in that the CGI performs all of the execution work and also has to be enabled in the WebSvcExtRestrictionList metabase property. Unlike ISAPI requests, CGI requests involve launching a new process for each request. However, this design does not perform or scale very well especially not on heavily trafficked Web sites. Consequently, CGI requests are typically used less often than other kinds of requests. Common Troubleshooting Tasks Several basic IIS troubleshooting tasks are helpful in many different troubleshooting situations. These tasks include: disabling HTTP friendly error messages in Internet Explorer, generating a custom HTTP request, checking basic request functionality, browsing with different host name styles, using substatus and Win32 errors in W3C extended logging, configuring the WWW service to log worker process recycling events, checking WindowsNT system and application event logs, checking the HTTP error and IIS logs, restarting the IIS service, and identifying the process ID of the worker process. Perform these tasks during the initial steps of the discovery phase, or when initially diagnosing a problem. Disabling HTTP Friendly Error Messages in Internet Explorer Internet Explorer displays friendly error messages when it encounters a problem during communication with an HTTP server. Although friendly error messages are appropriate for the end user, they prevent you, the administrator, from seeing the literal HTTP error codes and explanations, which makes troubleshooting difficult, if not impossible. During the discovery phase of troubleshooting, you can gather and record the HTTP errors that have occurred if you have disabled friendly messages. To disable friendly HTTP error messages in Internet Explorer On the Tools menu in Internet Explorer, click Internet Options. Click the Advanced tab. In the list of options under Browsing, clear the Show friendly HTTP error messages check box. Click Apply, and then click OK. Generating an HTTP Request To observe the raw HTTP traffic between a browser and IIS, use WFetch. Commonly, you might use WFetch when a user receives an HTTP 403-Forbidden error. In such a situation, you might begin troubleshooting by checking the access control list (ACL) configuration on the requested content. If that does not reveal the cause, you most likely would start troubleshooting the problem as an authentication problem. However, the 403 message indicates that these are the wrong actions to take. Disabling friendly HTTP error messages in Internet Explorer and re-trying the request with WFetch would have yielded a much more meaningful error: Directory Listing Denied. This virtual directory does not allow contents to be listed. You would then immediately realize that Directory Browsing is disallowed for that virtual directory and that either Default Documents is disabled or it is enabled, but none of the configured documents are present in that directory. To generate a simple HTTP request using WFetch Start WFetch. From the Verb list box, click GET. In the Host box, type a host name for example, www.contoso.com. In the Path box, type the following: / In the Auth list box, click Anonymous. The Port, Ver (HTTP Version), and Connect settings can be left at their defaults. Click Go. WFetch sends an HTTP request to the local instance of IIS and then receives the response. In the Log Output window, you can view the HTTP request and response data. Notice the individual headers in both the request and response, and the entity body in the response. Figure11.1WFetch Startup Screen. shows WFetch in the default startup state. Figure11. SEQ Figure \* ARABIC 1WFetch Startup Screen Checking Basic Functionality with Test Request Files Occasionally, you might need to assess the core functional health of the various request types in IIS. Determining which kinds of requests work and which do not is a fundamental step in the isolation phase. The easiest way to accomplish this is to create several test files that can be placed in various application roots in your various Web sites. Making these test requests from different locations is as important as making requests of different types. Consider that a directory location determines which application, URL, and application pool a request will be routed through. These locations can indicate at which point a particular kind of request is failing. The following examples are test request files for static content, ASP ISAPI, and ASP.NET ISAPI requests, respectively. Static Content Test Request File To create a static content test request file, create an HTML file with the following HTML code:") Response.Write ("The authenticated user is (blank if anonymous) : " & _ Request.ServerVariables("LOGON_USER") & _ "
") Response.Write ("The application pool that served this request is: " & _ Request.ServerVariables("APP_POOL_ID") & _ "
") %> Save this code in a file called Simple_Request.asp. ASP.NET Test Request File The code for an ASP.NET test request is similar to that of an ASP test request: <%@ Page Language="VB" Debug="True" Trace="True" %> <%@ Import Namespace=System.Web%>
Save this code in a file called Simple_Request.aspx. Testing Simple Object Creation A simple COM or .NET object that returns a basic hello world string is a useful troubleshooting tool. Placing that object locally on the server running IIS or remotely on your application tier, and then testing from ASP or from a simple script file or compiled client can help you determine where problems exist in your distributed applications architecture. Testing Application Code Outside of IIS Another way to test basic functionality is to test your applications completely outside of IIS. For example, if you are testing COM object creation failures, you can create a simple Microsoft VisualBasic Scripting Edition (VBScript) file that contains your object creation code and then run the script when you log on to the IIS Web server. Variations might involve creating other custom applications by using other development tools. Browsing with Different Host Name Styles Connectivity and authentication problems can be related to Domain Name System (DNS) issues, the configuration of host headers, and general network anomalies. To understand the nature of a problem, it is sometimes helpful to vary the way that you type the host name in the requested Uniform Resource Locator (URL). Use any of the following variations: Fully Qualified Domain Name (FQDN). A fully qualified DNS name that uniquely identifies the computer. This name is a combination of the DNS host name and the DNS domain name, in the format HostName.DomainName. For example, if the DNS host name is host1 and the DNS domain name is example.contoso.com, the fully qualified DNS name is host1.example.contoso.com. Network BIOS (NetBIOS) name. The computer name recognized by Windows Internet Name Service (WINS) in the format HostName, which maps the name to an IP address. For example, the NetBIOS name for host1.example.contoso.com is host1. IP address. The address used to identify a unique node on an IP network. This address is typically represented with the decimal value of each octet, separated by a period (for example, 192.168.7.27). Note IIS6.0 also supports IPv6 addressing. For more information, see IPv6 and IIS6.0iisrg_ip6_XJRF in this book. Browsing with different host name styles is useful for the following reasons: Certificate authentication requires matching the host name from the URL to the Common Name (CN) imprinted in the server certificate. In Kerberos authentication, the requested Kerberos service ticket is keyed off of the host name entered in the URL. If the wrong host name is used, a service ticket might not be obtainable. The host header configuration in the ServerBindings metabase property dictates how an HTTP request is routed into the service. This relies upon matching the requested host name with an appropriate host header configuration. The simple name resolution process can differ when you use a NetBIOS name instead of an FQDN. Using a NetBIOS name in the URL is similar to pinging to test name resolution of the NetBIOS name to the IP address. Internet Explorer can unexpectedly prompt for credentials during NTLM authentication (also referred to as WindowsNT Challenge/Response authentication) when the host name portion of the requested URL contains a period. Using the NetBIOS name rather than an FQDN or IP address might indicate the cause. For more information, see article 258063, Internet Explorer May Prompt You for a Password in the Microsoft Knowledge Base. To find this article, see the Microsoft Knowledge Base link on the Web Resources page at http:// HYPERLINK "http://www.microsoft.com/windows/reskits/webresources" www.microsoft.com/windows/reskits/webresources. Browsing by IP address is a foolproof way to rule out name resolution problems. However, the host header (set with the ServerBindings property) must be appropriately configured. Using Substatus and Win32 Errors in W3C Extended Logging To reduce the attack surface of IIS6.0, custom error messages do not return specific error message information, including the substatus code, to remote client computers. If a custom error message contains too much information about the core Web server and an explanation of why a particular request failed to execute, malicious users can use the information to attack the Web server. As an example, an error code such as 404.2-Web Service Extension Lockdown Policy Prevents This Request is returned to a Web client as 404-File or Directory Not Found, and, thereby, gives the Web client no indication of why the request failed. Although this change to error message reporting reduces the possibility of an attack by a malicious user, it makes it difficult to debug a failed request. For this reason, you can enable substatus error codes logging to IIS log files when appropriate. Note that on clean installs, substatus code logging for W3C Extended format files is enabled by default. Doing so is particularly useful when the logging of Win32 error codes is enabled. Then, to obtain the information that you need, you simply locate the specific request in the log file and check the Win32 error code and substatus code. This feature is called W3C extended logging. Figure11.2Extended Logging Properties Sheet shows substatus codes as they appear on the extended logging properties sheet. Figure11. SEQ Figure \* ARABIC 2Extended Logging Properties Sheet Important You must be a member of the Administrators group on the local computer to perform the following procedure or procedures, or you must have been delegated the appropriate authority. As a security best practice, log on to your computer by using an account that is not in the Administrators group, and then use the runas command to run IIS Manager as an administrator. At a command prompt, type runas /User:Administrative_AccountName mmc %systemroot%\system32\inetsrv\iis.msc. T o log substatus error codes In IIS Manager, double-click the local computer, double-click the Web Sites folder, right-click the Web site, and click Properties. On the Web Site tab, ensure that the Enable logging check box is selected. In the Active log format list box, click W3C Extended Log File Format. Click Properties. Click the Advanced tab, and under Extended properties, select the Protocol Substatus check box. Click OK twice. Configuring the WWW Service to Log Worker Process Recycling Events Proper configuration of IIS is always essential to successful troubleshooting. By setting the LogEventOnRecycle metabase property, you can direct the WWW service to log worker process recycling events in the event log. To accomplish this, you can set the LogEventOnRecycle property directly as a bitmask or you can set individual AppPoolRecycle property settings to true. The LogEventOnRecycle property is set to AppPoolRecycleMemory by default. This is important as there could be occasions when the problem reported happened at or near the same time as a worker process recycle for the application. Having this information makes it easier to troubleshoot exactly what happened at the time of the problem to the entire server. Table11.1 summarizes the event IDs that are logged, and the related metabase properties and their bitmasks. Table11. SEQ Table \* ARABIC 1LogEventOnRecycle Configuration Recycle ConditionEvent IDMetabase PropertyBitmaskElapsed time1074AppPoolRecycleTime1 (0x1)Number of requests processed1075AppPoolRecycleRequests2 (0x2)Scheduled recycle1076AppPoolRecycleSchedule4 (0x4)Virtual memory consumed1077AppPoolRecycleMemory8 (0x8)ISAPI reported itself unhealthy1078AppPoolRecycleIsapiUnhealthy16 (0x10)Recycled on demand1079AppPoolRecycleOnDemand32 (0x20)Recycled due to configuration change1080AppPoolRecycleConfigChange64 (0x40)Private memory consumed1117AppPoolRecyclePrivateMemory 128 (0x80) Checking NT System and Application Event Logs When looking for information about a problem, always consult the event logs for the application. To access the event logs From the Start menu, point to Administrative Tools, and click Event Viewer. In the console tree, click one of the three log types: Application, Security, or System. Checking the HTTP Error Log In IIS6.0, HTTP.sys logs errors in the HTTP error log. The HTTP error log is the first place you should look when IIS returns an HTTP 503Service Unavailable error, for example. The HTTP error log is located in systemroot\System32\LogFiles\HTTPERR. For information about HTTP errors, see HTTP Status Codesiisrg_trb_YBCG later in this chapter. Checking the IIS Logs The IIS logs provide a great deal of information about the activity of a Web application. You can find the IIS logs in systemroot\System32\LogFiles\W3SVCnumber, where number is the site ID for the Web site. Restarting IIS Services During troubleshooting, you might need to restart the IIS services. You can use IIS Manager to start or to stop any Web site on your server running IIS, to stop or recycle an application pool, or to restart the WWW service, which restarts the complete IIS service base. Restarting a Web Site Stopping a Web site causes the WWW service to remove the URLs for that site from the HTTP.sys configuration. The WWW service then marks the Web site as stopped in the metabase and directs the Web site not to start again automatically when the WWW service is restarted. Stopping a Web site does not cause loaded segments of the applications to unload, nor does it affect the worker processes that might be running when you stop the site. Additionally, requests that are waiting in the application pool queue in HTTP.sys will still be processed. However, new requests will either generate HTTP 404-Not Found errors, or, if no other Web site is listening on the port, will send the client a socket connection error. Web sites can be stopped and started by using either IIS Manager or the IisWeb.vbs command-line script. To stop and restart a Web site by using IIS Manager In IIS Manager, double-click the local computer, double-click Web Sites, right-click the Web site that you want to restart, and click Stop. (Stopped) appears next to the site name in IIS Manager. To restart the Web site, right-click the Web site, and click Start. To stop and restart a Web site by using IisWeb.vbs To stop a Web site, at a command prompt, type the following: iisweb.vbs /stop WebSiteName To restart the Web site, at a command prompt, type the following: iisweb.vbs /start WebSiteName Recycling an Application Pool Application pools are made up of a listening and routing structure in HTTP.sys and one or more ready-to-start worker processes that are waiting to process incoming requests. In worker process isolation mode, you can configure IIS to periodically restart worker processes under certain conditions. This automatic restarting of worker processes is called recycling. Recycling an application pool causes the WWW service to shut down all running worker processes that are serving the application pool, and then start new worker processes. Whether the WWW service starts the new worker processes before it stops the existing one depends on the DisallowOverlappingRotation property in the metabase. Recycling an application pool does not alter any state in HTTP.sys or change any configuration in the metabase. Note When an application pool is serviced by more than one worker process, the recycle operation is staggered in some scenarios. Only when a worker process is recycled on demand or when all of the worker processes hit their memory limits at the same time would they be restarted simultaneously. To recycle an application pool on demand In IIS Manager, double-click the local computer, double-click Application Pools, right-click the application pool that you want to restart, and click Recycle. For more information about when to manually recycle an application pool or worker process, see Recycling Worker Processes in IIS6.0 Help. Stopping an Application Pool Stopping an application pool causes the WWW service to shut down all running worker processes serving that application pool. HTTP.sys is then directed to return HTTP 503-Service Unavailable errors for any requests currently queued for this pool, and to return HTTP 503 errors for all future requests. The application pool is marked as stopped in the metabase and is configured not to start automatically when the WWW service is restarted. You must manually start the application pool to cause it to serve requests again. Restarting the WWW Service Sometimes, as a last resort, you need to restart the WWW service. To restart the WWW service In IIS Manager, right-click the local computer, point to All Tasks, and click Restart IIS. Click Restart Internet Services on servername, where servername is the name of your server. Click OK. Identifying Worker Process Process ID The nature of some problems, especially those in which an application is hanging, leaking memory, or generally misbehaving, make it necessary to know which application pool goes with which worker process (or processes, if Web gardens are enabled). With the advent of the new worker process isolation mode in IIS6.0, a Web server potentially can have thousands of individual instances of W3wp.exe running at any given time. Looking for the process ID in Task Manager is pointless in such a case because it is impossible to differentiate between the different worker processes servicing the application pools. In earlier versions of IIS and when IIS6.0 is running in IIS5.0 isolation mode, identifying the process ID is relatively easy. If a component installed in a COM+ library application has been activated from a worker process, the procedure for identifying the process ID is the same. To find the process ID in versions prior to IIS6.0 and in IIS5.0 isolation mode From the Start menu, point to Administrative Tools, and click Component Services. In the console tree of the Component Services window, double-click Component Services, double-click Computers, double-click My Computer, and double-click Running Processes. Under this node, you see the various COM+ applications and their associated process IDs. xe "aa \ \ i i s R G _ T R B . d o c - 1 1 2 1 " I n w o r k e r p r o c e s s i s o l a t i o n m o d e , y o u c a n f i n d t h e p r o c e s s I D b y u s i n g t h e c o m m a n d - l i n e s c r i p t I i s a p p . v b s , w h i c h i s s t o r e d i n s y s t e m r o o t \ S y s t e m 3 2 , a n d i s d e s i g n e d s p e c i f i c a l l y f o r t h i s p u r p o s e . I m p o r t a n t Y o u m u s t b e a m e m b e r o f t h e Administrators group on the local computer to run scripts and executables, or you must have been delegated the appropriate authority. As a security best practice, log on to your computer by using an account that is not in the Administrators group, and then use the runas command to run your script or executable as an administrator. At a command prompt, type runas /profile /User:MyMachine\Administrator cmd to open a command window with administrator rights and then type cscript.exe ScriptName (including the full path with parameters, if any). To find a worker process ID by using Iisapp.vbs Run Iisapp.vbs with no parameters. This displays a listing of running worker processes, their process IDs, and the application pool names. For more information about using Iisapp.vbs, see Listing Running Web Applications in IIS6.0 Help. HTTP Status Codes HTTP status codes provide the best information about what is happening with IIS. On an active Web server, HTTP status codes provide a continuous feed of up-to-the-moment information about the activity and health of the server. Most HTTP status codes have three digits, for example, 401. Some status codes have three digits followed by a decimal point, and one or two more digits (for example, 300.12). In such a case, the number that follows the decimal point is called the substatus code. Codes within each hundred-number range provide specific types of information, as summarized in Table11.2HTTP Status Codes. Table11. SEQ Table \* ARABIC 2HTTP Status Codes Status Code RangeType of Code100Informational200Successful300Redirection 400Client Error500Server Error As an administrator, at some point you will inevitably need to determine which HTTP status codes are being returned by IIS. There are a number of ways to do this. Your choice of method depends on the amount of information that you need and the type of problem that you are troubleshooting. If you have a problem that is limited to one or a few clients, viewing the status codes from the clients Web browser or issuing a reproduced HTTP request with WFetch might best serve your needs. However, if you need to look for an individual request or for a collection of multiple requests over a period of time, you should examine the IIS logs. Viewing IIS logs allows you to see not only the main HTTP status code, but the Win32 error code and the HTTP substatus code (if logging is configured to do so). The Win32 error codes and the HTTP substatus codes often contain information that is critical to your troubleshooting effort. The following sections describe the HTTP status codes and provide troubleshooting suggestions. HTTP 1xx-2xx Informational and Success Codes HTTP status codes in the 1xx are typically informational. Status codes in the 2xx range indicate that the transaction was successful. Because the codes in both ranges are self-explanatory, they do not require further discussion. HTTP 3xx Redirection Codes HTTP status codes in the 3xx range pertain to redirection (telling the browser to issue a new request at a different location). These codes do not indicate that a problem has occurred. Table11.3HTTP 300-Class Error Codes Returned by IIS lists the HTTP redirection codes that IIS returns. Table11. SEQ Table \* ARABIC 3HTTP 300-Class Error Codes Returned by IIS Status CodeCondition301Permanent Redirect.302Object Moved304Not Modified.307Temporary redirect. 301-Permanent Redirect This redirection status code tells the Web client that the location for the requested resource has permanently changed. The new location is specified in the Location HTTP header. This is primarily useful for Web clients that keep record of HTTP URL links. Upon receiving this status code, the Web client can update the stored URL with the new location. 302-Object Moved This standard redirection status code tells the Web client to issue a new request to the location specified in the Location HTTP header. 304-Not Modified RFC 2616, Hypertext Transfer Protocol HTTP/1.1 provides functionality for conditional GET requests. For example, an HTTP client can specify an If-Modified-Since HTTP header that consists of a time and date stamp, which tells the server that it has a cached copy of the content being requested. If the request meets the condition specified in the If-Modified-Since header, the server will not return a body, but will return the 304-Not Modified result. 307-Temporary Redirect The temporary redirect option, which is new in IIS6.0, is used to prevent a Web browser from losing data when the browser issues an HTTP POST request. Normally, when a Web browser issues a POST request and then receives a 302-Object Moved redirect message from the Web server, the browser issues a GET request for the new location and loses the data in the POST request. With a 307 redirect, the browser reissues the POST request with the original data to the new location. For more information about temporary HTTP redirects, see HttpRedirect in IIS6.0 Help. Courtesy Redirects When a browser makes an HTTP request, the request sometimes takes the form http://example.contoso.com/vdir1. Notice that there is no trailing / at the end of the URL. In this case, vdir1 is a directory, instead of a file. As a courtesy to the Web browser, IIS responds to this request with a 302-Object Moved redirect message with the HTTP header: Location: /vdir1/. This is known as a courtesy redirect. Courtesy redirection can produce unexpected behavior when the Web server is behind a Network Address Translation (NAT) device a proxy server, for example. Suppose, for example, that the Web site is configured to listen on port 8080 and the NAT device is configured to forward HTTP requests on port 80 to the Web server on port 8080. In this configuration, the NAT device accepts the Web browsers request, but when the courtesy redirect occurs, IIS appends the port number to the host name. The NAT device needs to be aware of this and alter the response appropriately before returning it to the client. For more information, see article 298408, IIS Generates Courtesy Redirect When Folder Without Trailing Slash Is Requested and article 313074, How To: Redirect Browser Requests in Internet Information Services5.0 in the Microsoft Knowledge Base. To find these articles, see the Microsoft Knowledge Base link on the Web Resources page at http://www.microsoft.com/windows/reskits/webresources. HTTP 40x Client Error Codes HTTP status codes in the 4xx range indicate that a problem occurred with the request. Request problems can range from the request not meeting authentication requirements to the request being malformed. RFC 2616, Hypertext Transfer Protocol HTTP/1.1 refers to request problems as client errors because the requests originate from the client. Table11.4HTTP 400-Class Client Error Codes Returned by IIS lists the client error codes that IIS returns. The descriptions for most codes are self-explanatory. When additional information about a code is required, it is provided in one of the following sections. Table11. SEQ Table \* ARABIC 4HTTP 400-Class Client Error Codes Returned by IIS Status CodeCondition400Cannot resolve the request.401.xUnauthorized.403.xForbidden.404.xFile or directory not found.405HTTP verb used to access this page is not allowed.406Client browser does not accept the MIME type of the requested page.407Initial proxy authentication required by the Web server.412Precondition set by the client failed when evaluated on the Web server.413Request entity too large.414Request URL is too large and therefore unacceptable on the Web server.415Unsupported media type.416Requested range not satisfiable.417Expectation failed.423Locked error. HTTP 400-Cannot Resolve the Request IIS returns a 400 error code when the request is malformed. Though there are a variety causes for this error, the most common reason is that the request does not comply with RFC 2616. For example, if the request is made in HTTP 1.1, RFC 2616 requires that the request include a Host header. The simplest HTTP 1.1 request would look similar to the following: GET / HTTP/1.1 Host: example.contoso.com You can use WFetch to issue such a custom-built request. Most HTTP clients adhere to HTTP RFC 2616 and issue well-formed requests. However, if you are developing a custom HTTP client and are experiencing HTTP 400 problems, use WFetch to make test requests. To issue a simple HTTP 1.1 request using WFetch Start WFetch. From the Advanced Request list box, click Raw Request. In the box, type the following text, where ComputerName is the name of your server running IIS: GET / HTTP/1.1\r\n Host: ComputerName\r\n \r\n Ensure that the Host box contains the same ComputerName as noted above. Click Go. This sends the most basic HTTP 1.1 GET request to the server. HTTP.sys records some errors in its own log file. Table11.5HTTP.sys Logged Reason Codes for HTTP 400 Errors lists the HTTP.sys reason codes and explanations for each of the 400 errors. These errors are not recorded in the IIS logs. Table11. SEQ Table \* ARABIC 5HTTP.sys Logged Reason Codes for HTTP 400 Errors Reason CodeConditionBadRequestThe request could not be understood by the server because the syntax was incorrect.VerbInvalid verb.URLInvalid URL.HeaderInvalid header name.HostnameInvalid hostname.Invalid_CR/LFInvalid carriage return or line feed.NumberInvalid number.FieldLengthA header field in the request was too long.RequestLengthThe request length was too long. HTTP 401.x-Unauthorized Authentication is one of the first operations performed when an HTTP request is issued. Authentication is the process whereby IIS creates a user context for an HTTP request, typically by obtaining credentials from the HTTP client using a preconfigured authentication method, and then calling a Windows logon API using those credentials. The API then returns a security token for the request, which represents the user context. After the authentication process concludes, IIS determines where the HTTP request goes next based on the resource that is being requested. Regardless of this choice, IIS issues an authorization check against the requested resource. IIS checks to ensure that the user context associated with this request is allowed to make the request. Usually, IIS performs a file ACL check to authenticate the request. When IIS cannot authenticate a request, it returns a 401.x-Unauthorized code. The substatus codes provide detailed information about why the request failed, as shown in Table11.1LogEventOnRecycle Configuration. The descriptions for most substatus codes are self-explanatory. When additional information about a substatus code is required, it is provided in one of the following sections. Table11. SEQ Table \* ARABIC 6HTTP 401-Substatus Codes 401 Substatus CodeCondition1Access is denied due to invalid credentials.2Access is denied due to server configuration favoring an alternate authentication method.3Access is denied due to an ACL set on the requested resource.4Authorization failed by a filter installed on the Web server.5Authorization failed by an ISAPI/CGI application.7Access denied by URL authorization policy on the Web server. 401.1 and 401.2-Authentication Problems Typically, HTTP 401.1 and 401.2 errors are encountered when the authentication process fails in some way either because the mechanism that IIS used to obtain the credentials has failed or because the credentials themselves are invalid. This is an important distinction that is a critical to the problem isolation process. These errors tell you that, before performing any other function, IIS has failed to authenticate the user. For a brief overview of the authentication options available in IIS, see Managing a Secure IIS6.0 Solutioniisrg_sec_OVERVIEW in this book. Initial Isolation When you know that authentication has failed, gather the following information: The URL that is being requested. The authentication method that is configured for that node in the metabase. When you know the requested URL, you can use IIS Manager to verify the authentication methods that are enabled. Alternatively, you can use the Adsutil.vbs command-line script to check the AuthFlags metabase property. For information about the settings for this property, see AuthFlags in IIS6.0 Help. Multiple Authentication Methods Typically, initial HTTP requests to a server are anonymous. In most cases, a Web browser does not know to send authentication information, and, therefore, it issues a request without this information. If Anonymous authentication is enabled for the requested resource, IIS does not attempt to use other authentication protocols, but instead attempts to obtain a security token for the IUSR_ComputerName account by calling one of the Windows logon APIs. If this fails, then IIS looks to see if other authentication methods are enabled for the URL and sends back a 401.2 response to the browser that includes a list of WWW-Authenticate HTTP headers that specify the authentication schemes IIS is configured to accept for that URL. At this stage, typically it is up to the HTTP client to choose to continue the HTTP transaction, to pick a protocol from the list, and to proceed to authenticate with that protocol. If this process is successful, IIS then calls a logon API and obtains a security token that it can use to associate a user context with the HTTP request. LogonMethod Default Has Changed The LogonMethod metabase property designates the kind of logon token that IIS asks for when calling the Windows logon APIs for Anonymous or Basic authentication. In earlier versions of IIS, the default was set to 0, which caused IIS to request an INTERACTIVE token. This sometimes caused problems when the IUSR_ComputerName, or, in the case of Basic authentication, the user credentials did not have the Log on locally user right. In such a case, IIS returns a 401.1 error. In IIS6.0, the default value of LogonMethod has changed to 3, which evaluates to a NETWORK_CLEARTEXT token. This token is very similar to an INTERACTIVE token; however, it does not allow the user to physically log on to the system console and does not require the Log on locally user right. For more information, see Securing Sites with Web Site Permissions in IIS6.0 Help. No Authentication Methods Selected (AuthFlags=0) IIS returns a 401.2 error when no authentication methods are enabled. If you receive this error, ensure that an authentication method is enabled. Anonymous Accounts (IUSR_ComputerName) Attempting Sub-authentication Logon Receive the 401.1 Error: Logon Failed By default, the sub-authentication component, Iissuba.dll, is not enabled in IIS6.0. In earlier versions, Iissuba.dll allowed IIS to manage passwords on anonymous accounts, which created a potential security risk. In IIS6.0, you can use sub-authentication to manage passwords for anonymous accounts. To do so, your configuration must meet the following requirements: For applications for which you grant anonymous access, the worker process must run as LocalSystem. The sub-authentication component, Iissuba.dll, must be registered. The AnonymousPasswordSync metabase property must be enabled (set to true). The actions taken to meet these requirements are different for clean installations of IIS6.0 and upgrades to IIS6.0 than they are for installations of IIS with sub-authentication configured. For more information about configuring sub-authentication, see Anonymous Authentication in IIS6.0 Help. Additionally, when you encounter a 401.1 error, ensure that the account configured for anonymous authentication has not expired or is not locked out. Non-Anonymous authentication Initially, a Web browser issues an anonymous request. If Anonymous authentication is not enabled or if Anonymous authentication fails, IIS sends a list of supported authentication protocols as a group of WWW-Authenticate HTTP headers. At this point, the browser chooses the protocol to use, and then continues processing the request. Basic authentication Basic authentication is the simplest authentication protocol. Because of its simplicity, it is a good authentication method to enable temporarily when you are troubleshooting authentication problems. Use this authentication protocol if you want to quickly verify that a password is correct or that communication with a domain controller is working. If the user name is a domain account, IIS contacts a domain controller to verify the credentials. If this fails, IIS returns a 401.2 error. If communication with the domain controller is successful, but the user name or password is invalid, IIS returns a 401.1 error. Internet Explorer not IIS allows the user three connection attempts before rejecting the connection request and reporting an error to the user. When the Web server verifies that the user name and password correspond to a valid WindowsServer2003 user account, a connection is established. Integrated Windows authentication Integrated Windows authentication actually encompasses two different authentication protocols:- Kerberos and WindowsNT Challenge/Response, also known as NTLM. When this authentication protocol is enabled, IIS sends the following HTTP headers to the browser: WWW-Authenticate: Negotiate WWW-Authenticate: NTLM The Negotiate header indicates that IIS can negotiate which of the two Windows authentication protocols to use. Internet Explorer 5, or later, recognizes this authentication header and determines which of the two authentication protocols to use. The NTLM authentication header is used as a fallback for browsers that do not support the Negotiate authentication header. Kerberos protocol authentication Authentication with the Kerberos protocol occurs as follows: IIS requests authentication. If the Windows user has not logged on to a domain, a dialog box requesting credentials appears in Internet Explorer, and then IIS contacts the Ticket Granting Server or Key Distribution Center (KDC) to request and receive a Ticket Granting Ticket. It then sends the Ticket Granting Ticket along with information about the server running IIS to the KDC. If the Windows user has already successfully logged on to the domain and received a Ticket Granting Ticket, IIS sends this ticket along with information about the IIS server to the KDC. The KDC issues the client a Resource Ticket. The client passes this ticket to the server running IIS. When troubleshooting Kerberos, ensure that the browser can make a trusted connection to the KDC by meeting the requirements listed in article 326985, How To: Troubleshoot Kerberos-related Issues in IIS in the Microsoft Knowledge Base. To find these articles, see the Microsoft Knowledge Base link on the Web Resources page at http://www.microsoft.com/windows/reskits/webresources. Additionally, consider potential problems with the following: Service Principal Names. Kerberos uses a service for authentication. When the client passes the Ticket Granting Ticket and the information about IIS to the KDC, part of this information includes the server name from the URL that the browser is attempting to reach. By default, upon installation a systems NetBIOS name is registered in Active Directory directory service. This is the name that a KDC uses to recognize IIS. It is called the Service Principal Name (SPN). If the browser is accessing the Web server by using some other host name either the FQDN or another host header name that name must be registered with Active Directory. You can do this by using the Setspn.exe command-line utility. For more information about Setspn.exe, in Help and Support Center for WindowsServer2003, click Tools, and then click Windows Support Tools. Kerberos authentication. If you suspect that you are having a problem with the SPN, test Kerberos authentication by using the NetBIOS computer name. If the test is successful, you probably need to use the Setspn.exe command-line utility. To run Setspn.exe, at a command prompt, type the following: Setspn -A HOST/ServicePrincipalName ServiceAccount In the above syntax, ServicePrincipalName is the new host name that you want to register and ServiceAccount is the identity under which the process that is hosting the service will be running. For computer accounts, like NetworkService or LocalSystem, use the NetBIOS name of the computer. Note The LocalService account, along with any other local accounts on an IIS server, cannot be authenticated by a KDC. Therefore, you cannot use the Kerberos protocol for authentication if the worker process identity is set to LocalService or to another account on the local computer. Trust for delegation configuration. When troubleshooting Kerberos authentication, also ensure that the IIS computer is configured to be trusted for delegation in Active Directory. If the server running IIS is a member of the domain and is not a domain controller, then it must be trusted for delegation in order for Kerberos to work properly. To configure a computer to be trusted for delegation On the Domain controller, click Start, point to Administrative Tools, and click Active Directory Users and Computers. Under the domain, double-click the Computers folder, right-click the server that is running IIS, and then click Properties. Click the General tab. Select the Trusted for delegation check box. Click OK. The computer account is now trusted for delegation. Security auditing. By enabling security auditing on the server running IIS, you can use the events that are logged to discover whether problems are occurring. For information about using the security log to troubleshoot Kerberos, see article 326985, How To: Troubleshoot Kerberos-related Issues in IIS in the Microsoft Knowledge Base. To find these articles, see the Microsoft Knowledge Base link on the Web Resources page at http://www.microsoft.com/windows/reskits/webresources. WindowsNT Challenge/Response protocol WindowsNT Challenge/Response (NTCR) protocol differs from Kerberos in that the server presents the HTTP client with a challenge and the client responds with its response. This way, the clients password is never sent over the network. Authentication with the NTCR protocol occurs as follows: Typically, the client issues an initial anonymous request. When the anonymous request is rejected, IIS returns a 401.2 error and the WWW-Authenticate headers. If the client fails or does not support Kerberos, the Negotiate and NTLM header values initiate an NTCR authentication exchange. The client closes the TCP connection, opens a new one, and sends a request that includes an Authorization: NTLM header. This header also includes encoded text that represents the users UserName, ComputerName, and Domain. This text is used by the Windows Security Support Provider Interface (SSPI) to generate the challenge. If the user account is not a local Windows account on the IIS server, the data is passed on to an appropriate domain controller, which then generates the challenge. The challenge is sent to the client and IIS returns another 401.2 error. The client uses its password and the challenge to create a mathematical hash. The client sends the hash back to the server in another Authorization: NTLM header. The server accepts the response, and the local security provider or the appropriate domain controller recreates the same hash and compares the two. If they match, the user is successfully authenticated. When troubleshooting NTCR authentication, note the following: Point-to-point connections. NTCR authentication requires point-to-point connection between the client and the server that is running IIS. Consequently, the NTRC protocol will not work with some Conseil Europeen pour la Recherche Nucleaire (CERN) proxies and other Internet devices. As a result, NTCR is not suitable for use over the Internet. If a connection is broken during authentication, IIS returns a 401.2 error. HTTP keep-alives. A connection must be maintained through the entire authentication sequence. Therefore, HTTP keep-alives must be enabled on the server running IIS. If they are not enabled, IIS continues to return 401.2 errors until the browser fails the authentication sequence. Security privileges. The user account accessing the server must have the Access this computer from the network privilege. If this is not the case, IIS returns a 401.1 error. Domain controller communication. As with Basic authentication, if the credentials presented by the client are for a domain user account, the server running IIS needs to contact a domain controller. Note In earlier versions of IIS, to enhance performance, administrators could use the AuthPersistence metabase property to force IIS to negotiate one-time-per-client connections, and then use the same credentials for subsequent requests over the same connection. For security reasons, this feature has been removed from IIS6.0. Now the only valid setting for the AuthPersistence property is AuthPersistSingleRequest. 401.3-Unauthorized: Access Is Denied Due to an ACL Set on the Requested Resource A 401.3 error is a catchall access denied error. To troubleshoot this error, first check the authentication method that is enabled on the requested URL. This will help you determine the security context under which the request is expected to execute: as an anonymous user or as an authenticated user. How you proceed depends on whether the user is authenticated or not. Verify file and directory ACLs. When the requested URL is a directory, not a file, check the default document(s) that are configured for that URL. Sometimes, if the first document in the default document list is missing, the second document in the list might generate a 401.3 error. After verifying the ACLs, you have a few options. You can enable security auditing for object access failures. This can be difficult, though, because you have to enable auditing on the individual objects (files, directories, and so on) that you want to monitor. Alternatively, you can use File Monitor and Registry Monitor to look for access denied problems in the registry or file system. Simplified UNC authentication model The Universal Naming Convention (UNC) authentication method, also called UNC Pass-through authentication, determines the credentials used for gaining access to a UNC share on a remote computer. Beginning with IIS6.0, the UNC authentication method looks at the request user and the credentials stored in the UNCUserName and UNCPassword properties of the metabase to determine the credentials to pass through to the computer with the UNC share. This occurs in the following way: If the UNCUserName is specified (not empty) and UNCPassword is valid, the request user credentials are sent as the user identity for access to the remote share. If the UNCUserName is specified and the UNCPassword is not valid, a 500-Internal Server Error message is sent to the client. For information about troubleshooting this error, see HTTP 500.x-Internal Server Erroriisrg_trb_PKIX later in this chapter. If the UNCUserName is not specified, then the credentials of the request user (either an authenticated set of credentials for authenticated requests or IUSR_ComputerName credentials for anonymous requests) are sent as the user identity for access to the remote share. Note The UNCAuthenticationPassthrough metabase property is no longer used for UNC authentication. For more Information about authentication and authorization, see the following articles in the Microsoft Knowledge Base: Article 264921, Info: How IIS Authenticates Browser Clients Article 326985, How To: Troubleshoot Kerberos-Related Issues in IIS Article 294382, Authentication May Fail with 401.3 Error If Host Header Differs from Servers NetBIOS Name Article 811889, How To: Troubleshoot the Cannot Generate SSPI Context Error Message Article 325358, How To: Configure User and Group Access on an Intranet in Windows2000 or WindowsNT4.0 To find these articles, see the Microsoft Knowledge Base link on the Web Resources page at http://www.microsoft.com/windows/reskits/webresources. HTTP 403.x-Forbidden Typically, 403 errors occur when an operation or request is disallowed because a requirement other than proper authentication credentials is not met. Commonly, this error occurs when a script request is made to a Web directory for which script access is not enabled. In such a case, first verify for which resource the request is being made. If the request is, indeed, for script code, such as ASP or ASP.NET, check the execute permissions for that directory in IIS Manager and ensure that at least Script is selected. Table11.7 HTTP 403 Substatus Codes lists the HTTP 403 error substatus codes. Table11. SEQ Table \* ARABIC 7 HTTP 403 Substatus Codes 403 Substatus CodeDescriptionNoneAccess is denied.1Execute access is denied.2Read access is denied.3Write access is denied.4SSL is required to view this resource.5SSL 128 is required to view this resource.6IP address of the client has been rejected.7SSL client certificate is required.8DNS name of the client is rejected.9Too many clients are trying to connect to the Web server.10Web server is configured to deny Execute access.11Password has been changed.12Client certificate is denied access by the server certificate mapper.13Client certificate has been revoked on the Web server.14Directory listing is denied on the Web server.15Client access licenses have exceeded limits on the Web server.16Client certificate is ill-formed or is not trusted by the Web server.17Client certificate has expired or is not yet valid.18Cannot execute requested URL in the current application pool.19Cannot execute CGIs for the client in this application pool.20Passport logon failed. HTTP 404.x-File or Directory Not Found A HTTP 404 error indicates that the requested resource was not found. Table11.8HTTP 404 Substatus Codes lists the substatus codes for the 404 error. The descriptions for most substatus codes are self-explanatory. When additional information about a substatus code is required, it is provided in one of the following sections. Table11. SEQ Table \* ARABIC 8HTTP 404 Substatus Codes 404 Substatus CodeConditionNoneFile or directory not found.1Web site not accessible on the requested port.2Web service extension lockdown policy prevents this request.3MIME map policy prevents this request. Whether the requested resource is a file or a directory, begin by checking the following potential causes: Verify the file or directory. Examine the requested URL and check the physical path to which it maps. Check for unexpected host header routing. Problems with host headers occur when you have created more than one virtual Web server and rely on host header routing to serve content from the appropriate server. A 404 error results when you request a URL that exists on only one virtual server and, because of the configuration of the host header, the request is routed to the other virtual server. Verify that the request has been made on the correct port. The 404.1 error is returned when a request is made to a port on which the WWW service is not listening. Ensure that the requested file is not hidden. IIS sends a 404 error if the Hidden file attribute for a requested file is set. Validate (*ScriptMaps) Web service extension requests. 404 errors are sometimes returned when you have a *ScriptMap Web service extension and the Verify that file exists option is set to true. Because no file is actually associated with a *ScriptMap request, IIS cannot check for the file, so it returns a 404 error. Both the Web service extension lockdown policy and MimeMap restriction features can cause a 404 error to be returned to the client making the request. If one of these features is the cause of an HTTP 404 error, do the following: Check the sc-substatus field for the request in the IIS W3C extended log. For information about enabling substatus logging, see Using Substatus and Win32 Errors in W3C Extended Loggingiisrg_trb_ONKR earlier in this chapter. Check the Win32 error in the IIS W3C extended log. By default, IIS logs all Win32 errors associated with the request that are returned by the underlying operating system. Enable a modified custom error for the HTTP 404 error that you suspect. You can configure IIS to return a response to the client that contains the substatus code by editing the custom error page for the 404 error, and placing some custom text into the file. The custom error pages are located in systemroot\Help\IisHelp\Common. Important Enabling a custom error that provides substatus information undermines the security strategy used by IIS and should be done temporarily and only for troubleshooting purposes. 404.1-Web Site Not Accessible on the Requested Port The 404.1 error can occur only on computers with multiple IP addresses. If a specific IP address/port combination receives a client request, and the IP address is not configured to listen on that particular port, IIS returns a 404.1 HTTP error. For example, if a computer has two IP addresses and only one of those IP addresses is configured to listen on port 80, any requests received on the other IP address with port 80 causes IIS to return a 404.1 error. This error should be set only at the service level because it is returned to clients only when multiple IP addresses are used on the server. 404.2-Lockdown Policy Prevents This Request If a request is denied because the associated ISAPI or CGI has not been unlocked, a 404.2 error is returned. When substatus logging is enabled, if you look at this request in the IIS logs, you will see an entry similar to the following: 2002-11-25 05:46:15 127.0.0.1 GET /default.asp - 80 - 127.0.0.1 - 404 2 1260 If IIS logging is not configured to log the substatus code, you can check the Win32 error to verify this condition. When a 404.2 error occurs, IIS logs the Win32 error 1260, which is an ERROR_ACCESS_DISABLED_BY_POLICY error. For information about how to enable an application that has been locked down, see IIS6.0 Administrative Scripts, Tips, and Tricksiisrg_adm_NAPF in this book. 404.3-MIME Map Policy Prevents This Request If a request is denied because a MIME map restriction is in effect, a 404.3 error is returned. When substatus logging is enabled, if you look at this request in the IIS logs, you will see an entry similar to the following: 2002-11-25 05:46:27 127.0.0.1 GET /somefile.unkext - 80 - 127.0.0.1 - 404 3 50 As with a 404.2 error, a 404.3 error causes a Win32 error to be logged. In this case, the error is 50, which is an ERROR_NOT_SUPPORTED error. Another quick way to verify that a MIME map restriction is in effect is to add a wildcard MIME type to the virtual directory in question. This enables all MIME types to be served without restriction. Then, repeat the failed HTTP request and verify that it is served successfully. Note Adding a wildcard MIME type to the virtual directory is a troubleshooting step, not a solution to the problem. Leaving the wildcard in place and allowing all MIME types to be served compromises the security of the server. For more information about adding a wildcard MIME type, see Working with MIME Types in IIS6.0 Help. 405-HTTP Verb Used to Access This Page Is Not Allowed This HTTP code is returned when the client makes an HTTP request that contains a verb that is not allowed. This condition can occur when: A request for static content contains verbs other than GET or HEAD, and the request is made to a URL that did not end with a /. Instead performing a courtesy redirect, IIS sends the 405 error. An HTTP request for an ISAPI application contains a verb not listed in the ScriptMaps configuration for that ISAPI. 407-Initial Proxy Authentication Required by the Web Server This error indicates that an intermediary proxy server between the HTTP client and the Web server requires some form of authentication. How you troubleshoot this kind of error depends upon the proxy server itself. Generally speaking, running a network trace with Network Monitor is helpful. If the Web client is a custom client, ask its developer to ensure that it is handling security appropriately. 413-Request Entity Is Too Large For security reasons, you can limit the size of the entity-body of an HTTP request by modifying the MaxRequestEntityAllowed metabase property. When an entity-body of a client request exceeds the size that is specified in the MaxRequestEntityAllowed property, IIS returns a 413 error. If this error is logged for an individual request, an application on the Web server might have encountered an unexpected event and generated a request that is too large. If this error is logged for many requests, a malicious user might be attempting to compromise your Web server. 414-Request URL Is Too Large and Therefore Unacceptable on the Web Server Just as the entity body of a request can be too large for IIS to process, a URL can be too long for IIS to process. IIS returns a 414 error if this occurs. HTTP 5xx Server Error Codes Error codes in the 5xx range indicate server problems. Table11.9HTTP 500-Class Client Error Codes Returned by IIS lists the HTTP 5xx-class codes that IIS returns. The descriptions for most codes are self-explanatory. When additional information about a code is required, it is provided in one of the following sections. Table11. SEQ Table \* ARABIC 9HTTP 500-Class Client Error Codes Returned by IIS Status CodeCondition500.xInternal server error.501Header values specify a configuration that is not implemented.502Bad gateway.503Service unavailable.504Gateway timeout.505HTTP version not supported. HTTP 500.x Internal Server Error Codes Table11.10HTTP 500 Substatus Codes lists the most common 500 substatus codes returned by IIS. The descriptions for most substatus codes are self-explanatory. When additional information about a substatus code is required, it is provided in one of the following sections. Table11. SEQ Table \* ARABIC 10HTTP 500 Substatus Codes 500 Substatus CodeCondition11Application is shutting down on the Web server.12Application is busy restarting on the Web server.13Web server is too busy.14Invalid application configuration on the server.15Direct requests for Global.asa are not allowed.16UNC authorization credentials are incorrect.17URL authorization store cannot be found.18URL authorization store cannot be opened.100Internal ASP error. 500.11 and 500.12-Application State Issues Users rarely encounter 500.11 and 500.12 errors in their browser. These errors indicate that the ASP application to which the requested ASP page belongs is in the process of shutting down, restarting, or both. Restarts and shutdowns can be triggered directly from code when the built-in APIs available to ASP developers are used. Occasionally, these errors occur when the Global.asa file for the application (if one exists) is updated or altered in such a way that a file change notification is issued. Because IIS monitors for file change notifications on ASP content, IIS receives this notification, and because the file is the Global.asa file, IIS restarts the application. If users often encounter these errors in their browser and you are certain that the code in the Global.asa file is not being updated, virus scanning and backup software, which sometimes sweep the ASP content and trigger a notification, might be causing these errors. For more information, see article 248013, Err Msg: HTTP Error 500-12 Application Restarting in the Microsoft Knowledge Base. To find this article, see the Microsoft Knowledge Base link on the Web Resources page at http://www.microsoft.com/windows/reskits/webresources. 500.13-Web Server Too Busy ASP requests are executed by threads from a pool of threads whose size is limited to 25 threads per processor, by default. If more ASP requests arrive than there are threads available to execute them, ASP places the extra requests in a queue, where they wait until a thread becomes available. By default, this queue is limited to 3000 requests. If the queue becomes full, the next ASP request, instead of being queued, causes a 500.13-Web server too busy error. The number of requests that can wait in the queue is controlled by the AspRequestQueueMax metabase property. Change the AspRequestQueueMax property to produce the desired user experience a smaller queue allows users to more quickly see the 500.13 error when ASP requests are backed up. If users encounter this error, select the Active Server Pages/Requests Queued performance counter check box. If the queue fills up, but ultimately empties again, or if the queue remains partially filled but requests move through very quickly, there probably is not a problem, just a spike in traffic. However, if the queue remains partially or completely full for a sustained period and requests dont appear to be moving through quickly enough, there might be a problem. Diagnosing a problem of this type can be a complex process. For information about diagnosing such problems, see Advanced Troubleshootingiisrg_trb_KMFN later in this chapter. 500.16-UNC Authorization Credentials Are Incorrect Problems with content stored on remote file servers can be more complex than problems with locally stored content. The 500.16 error indicates that there is a specific configuration problem with remotely stored content. When IIS is configured to use specific credentials to access the UNC path in the metabase, it uses those credentials to open the content on the remote file server. To do this, IIS uses a Windows logon API to obtain a security token that it can use to impersonate a security identity when accessing the remotely stored content. If the call to the Windows logon API fails, indicating that there is a problem with the user name or password, IIS returns an HTTP 500.16 error. 500.17 and 500.18-IIS URL Authorization Issues IIS URL Authorization is a security feature that is new in IIS6.0. Among other administrative tasks, IIS URL Authorization configures the Authorization Manager store by setting the AzStoreName metabase property. This store can be an XML file or an LDAP location. It is configured by setting the AzStoreName property to an appropriate string that represents the path to a store. For example: msxml://c:\MyStore.xml or msldap://CN=MyAzStore, CN=Program Data, DC=MyDomain, DC=com. The 500.17 and 500.18 errors pertain to problems accessing the Authorization Manager store. A 500.17 error indicates that the store could not be found. To begin troubleshooting one of these errors, try to verify the path specified in the string by either opening the XML file or testing the LDAP path with a simple client, such as LDP.exe. For more information about LDP.exe, in Help and Support Center for WindowsServer2003, click Tools, and then click Windows Support Tools. A 500.18 error typically indicates that the store was found, but could not be opened. The error description might contain information such as an HRESULT and description. However, this error generally occurs because there is an ACLrelated problem. The security context that is used to open the store is that of the worker process identity. To troubleshoot, verify that the configured worker process identity is a member of the built-in IIS_WPG group and that this group has access to the store. File Monitor is a good utility to use in such a case. For more information about IIS URL Authorization, see URL Authorization in IIS6.0 Help. 503-Service Unavailable HTTP 503 errors are returned more often in IIS6.0 than in earlier versions of IIS. Typically, the HTTP 503-Service Unavailable error is returned directly from HTTP.sys when there is a problem with getting the request up to the worker process. However, this error can occur for a variety of reasons, as listed in Table11.11HTTP 503 Errors from HTTP.sys. Table11. SEQ Table \* ARABIC 11HTTP 503 Errors from HTTP.sys Reason StringText in HTTP.sys Error LogCause(s) of ErrorService unavailable.N/AIIS cannot start any new worker processes because of limited system resources or because starting a new worker process would exceed the DemandStartThreshold property. Bandwidth throttling is enabled, but the filter addition fails. The control channel or internal configuration group for the URL is inactive. The send for a request that was serviced from the cache failed (typically under low memory conditions).Too many users.ConnLimitThe connection limit for the site or server as a whole has been reached.Application taken offline.AppOfflineThe application pool has been put into Rapid Fail Protection and has been disabled automatically.Application request queue full.QueueFullThe application pools request queue has been exceeded and the request cannot be queued.Administrator has taken application offline.DisabledThe administrator has stopped the application pool.Application automatically shut down due to administrator policy.AppShutdownThe application pool has been shut down because it exceeded its CPU usage limit.Request timed out in app pool queue.AppPoolTimerThe application pool is too busy to handle the request. The request has, therefore, timed out in the queue and has been returned with a 503 error. For security purposes, the description that is sent back to the client (Service Unavailable) does not fully describe the error. HTTP.sys keeps its own error log. Requests that generate 503 errors never pass into user mode; therefore, the HTTP error log is the only place these problems are documented. To troubleshoot the cause of the 503 error, use the following procedure. To check for possible causes of HTTP 503 errors Check the HTTP.sys error log for information about the error being returned. See Error! Reference source not found. for error descriptions. Check the event logs. Typically, the event logs contain more detailed information about the cause of a 503 error. Verify that health/recycling issues are not causing the error. Check for server binding conflicts. Check the IP inclusion list for errors. Other HTTP.sys Error Log Errors In some cases, HTTP.sys generates entries in its error log that are not associated with a specific HTTP request and that do not have HTTP error codes. Table11.12Other HTTP.sys Errors lists these errors. Table11. SEQ Table \* ARABIC 12Other HTTP.sys Errors Logged Reason CodeConditionTimer_ConnectionIdleThe connection was reset as its idle time exceeded the ConnectionTimeout property.Timer_HeaderWaitThe headers for the request were not received within the time limit specified by the HeaderWaitTimeout property.Timer_MinBytesPerSecondThe throughput rate of this connection has dropped below the limit specified in the MinFileBytesPerSec property.Timer_EntityBodyThe entity body was not received within the specified timeout.Timer_ResponseThe connection was reset because the response was not processed within the specified time limit.Timer_AppPoolThe connection was reset because the request stayed in the request queue of the application pool longer than the specified time limit. For complete information about the errors logged in the HTTP error log, see the topic, Error Logging in the HTTP API on MSDN. To find this information, see the MSDN Library link on the Web Resources page at http://www.microsoft.com/windows/reskits/webresources. For further information about available configuration options in HTTP.sys, see article 820129, Inf: Http.sys Registry Settings for IIS in the Microsoft Knowledge Base. To find this article, see the Microsoft Knowledge Base link on the Web Resources page at http://www.microsoft.com/windows/reskits/webresources. Troubleshooting Configuration Problems Configuration problems present their own unique troubleshooting challenges. This section provides information about troubleshooting common configuration problems that occur with the integrity of XML in the IIS metabase and with UNC content. Preserving the Integrity of XML in the IIS Metabase With the introduction of the XML-based IIS configuration store, administrators now have the ability to directly edit XML in a text editor. This feature is called edit-while-running. Because XML has very strict rules about syntax, the XML store needs to be well-formed at all times. Manually editing the XML store with a text editor can introduce syntax errors in the MetaBase.xml file. Therefore, when manually editing the MetaBase.xml file with a text editor, observe the following guidelines: If a syntax error in the XML file creates a problem, use the metabase backup/restore feature to restore the metabase from the backup files. Open the MetaBase.xml file in Internet Explorer, which has a built-in XML parser. If a syntax error occurs, Internet Explorer displays it and notes its location. For information about XML syntax, see Writing Changes to MetaBase.xml in IIS6.0 Help. For information about the metabase edit-while-running feature of IIS, see Enabling Edit-While-Running in IIS6.0 Help. For information about the metabase backup/restore feature, see Restoring the Metabase from Backup Files in IIS6.0 Help. Troubleshooting Problems with UNC Content Traditionally, situations that involve Universal Naming Convention (UNC) content have experienced configuration and performance problems related to security, user delegation, and file caching. Improvements in IIS6.0 affect how these problems are resolved. Figure11.3IIS6.0 UNC Authentication illustrates the logic IIS6.0 uses to manage and serve content from a UNC file share. Figure11. SEQ Figure \* ARABIC 3IIS6.0 UNC Authentication Accessing and Executing Remote Content When IIS accesses content stored on a remote file server, user delegation becomes a factor because IIS uses the clients credentials to access content on the file server. In effect, the Web client passes the users credentials to IIS, which then delegates them to the file server for the authorization check. In IIS6.0, user delegation is easier because, by default, IIS uses the credentials of the request user (the anonymous user identity for anonymous requests and the authenticated users identity for authenticated requests) to access the content on the remote file server. So, by default, IIS does pass-through authentication, which means that IIS passes the user credentials through to the remote file server. If you want IIS to use a specific user when accessing remote content, set the UNCUsername and UNCPassword properties in the IIS metabase. For information about setting these properties, see UNCUsername and UNCPassword properties in the Metabase Property Reference of IIS6.0 Help. Logging On In Figure11.3IIS6.0 UNC Authentication, the request user is the identity of the request coming into the Web server, such as a GET request for HYPERLINK "http://www.microsoft.com/staticfile.txt" http://www.contoso.com/staticfile.txt. If anonymous access is enabled, this request uses the identity of the AnonymousUser as specified in the metabase (typically, IUSR_MachineName). If anonymous access is disabled and a valid authenticated request comes in, IIS uses that users account to log on. However, in the case of UNC content, at this stage, IIS checks if UNCUserName is specified in the metabase. If so, IIS logs on as that user. If UNCUserName is not specified in the metabase or if it is set to an empty string, then IIS logs on as the user who originally made the request (that is, as an anonymous user or as an authenticated user). Note The UNCPassthroughAuthentication metabase property has been removed from IIS6.0. In IIS6.0, pass-through authentication is enabled by default. To disable pass-through authentication, use the UNCUserName and UNCPassword properties. Accessing the Remote Server After IIS has a user identity, it must access the remote file server. The type of authentication (Anonymous, Basic, NTLM, Kerberos) enabled on the server can significantly affect how successful this step is. In almost all cases, Basic authentication enables IIS to access a remote file server (although you must always use SSL with Basic authentication since user names and passwords are sent in plain text). Using Basic authentication can help you verify that everything else is working properly. Executing the Remote Content So far, we have discussed how remote content is accessed. Remote content must also be executed in the context of this user account. In other words, if UNCUserName and UNCUserPassword are set, then all remote content will be accessed and executed in the context of UNCUserName. This might not be desirable for most applications, because with this approach you lose the identity of the originating Internet/Intranet user. If pass-through authentication is being used, then the request is processed under the identity of the request user. The process illustrated in Figure11.3IIS6.0 UNC Authentication applies to all content that is executed or retrieved by the server. Consequently, static content, such as Graphics Interchange Format (GIF) and Hypertext Markup Language(HTML) pages, Script-Mapped files, CGI executables, and ISAPI Extension DLLs (like ASP, ASP.Net, and so on) runs under this user context. (ISAPI filters run under the context of the worker process identity. By default, the worker process runs as NetworkService, which is a local account.) Working in a Domain Environment In most cases, you will want to run your Web servers and file server in a domain environment because it is much simpler to use a domain environment in which you can control one set of users and groups from the domain controller. Furthermore, if you want to use Windows Integrated Authentication in this scenario, then you need to use a domain environment to take advantage of the new constrained delegation feature of WindowsServer2003. Instructions on setting up constrained delegation follow. For more information about constrained delegation, see Delegating authentication in Help and Support Center for WindowsServer2003. Working in a Workgroup Environment Web servers and a file server used in a workgroup environment are difficult to manage and maintain. In a workgroup environment, because one user account has access to another computer only if that computer has a user with the same user name and password, you have added administrative overhead. To add a new user to your environment, you have to add the user to both servers and you need to ensure that both accounts have the same password. For a complete look at using remote content on a UNC share, see the whitepaper Deploying and Configuring Internet Information Services (IIS)6.0 with Remotely Stored Content on UNC Servers and NAS Devices. You can find this article by seeing the UNC Servers Accessed by Using IIS6.0 link on the Web Resources page at http://www.microsoft.com/windows/reskits/webresources. Troubleshooting File Caching Problems There are two causes for problems with file caching of UNC content: Vendors implementation and support for file change notifications. Usually non-Windows-powered Network Attached Storage (NAS) vendors provide fixes for known problems; however, as a workaround, caching is typically disabled to prevent those problems. The number of simultaneous distinct file structures that the ASP and static file caches can monitor for changes. Because each monitor takes up a work item indefinitely from the Server Message Block (SMB) connection with the file server, IIS can exhaust a systems SMB resources by trying to cache a wide and disparate set of directories. To address these problems, several changes have been made to the way IIS caches content. These changes improve performance and reliability in most cases; however, these changes can have unexpected effects on UNC stored content. ASP and Static File Caching Changes IIS6.0 has a new caching algorithm for the ASP script and static file cache when the content is being stored on a UNC share. In earlier versions, IIS registered for a file change notification from the operating system for each directory structure so that it would be notified when a file changed. In IIS6.0, the Web server simply checks the Last-Modified date on the cached file instead of requesting these notifications. If the Last-Modified date has not changed, the cached version of the file is sent out. If the file is new, the cache is updated with the new version of the file and is sent to the client. To further improve performance, the algorithm has a staleness aspect that causes IIS to check the Last-Modified date only if more than five seconds have passed since the last check. Otherwise, IIS assumes that the file has not changed, and serves the cached version. For more information about this algorithm, see Troubleshooting Performance Problems with UNC Contentiisrg_trb_MUUM later in this chapter. Note The server-side includes file handler (Ssinc.dll) makes use of the static file cache, so you see this file caching behavior for files with .stm and .shtml file name extensions, and for any other files that you have mapped to Ssinc.dll Troubleshooting Problems with Stale Content When stale content is being served by the Web Server even though the content has been updated on the remote file server, disable the Web Server cache as the first step in isolating the problem. Note First ensure that your results arent being affected by the 5-second staleness factor. To disable the ASP file cache In IIS Manager, double-click the local computer, right-click Web Sites, and click Properties. Click the Home Directory tab. Click Configuration. On the Cache Options tab, click the Do not cache ASP files option. Click OK twice. To disable the static file cache Navigate to the following subkey: HKLM\System\CurrentControlSet\Services\Inetinfo\Parameters Add DisableMemoryCache, a DWORD value, and set it to 1. Caution Do not edit the registry unless you have no alternative. The registry editor bypasses standard safeguards, allowing settings that can damage your system, or even require you to reinstall Windows. If you must edit the registry, back it up first and see the Registry Reference on the Microsoft Windows Server2003 Deployment Kit companion CD or on the Web at HYPERLINK "http://www.microsoft.com/reskit" http://www.microsoft.com/reskit. Troubleshooting Performance Problems with UNC Content Using the Last-Modified time to check for changes (instead of monitoring each virtual directory for change notifications) permits greater scalability when many virtual directories are mapped to UNC shares. Checking the Last-Modified time on the remote file server every time IIS wants to send a file is relatively expensive in terms of processing overhead. IIS checks the Last-Modified time on a file every five seconds by default; otherwise, it assumes that nothing has changed and serves the content from the cache, even though it might be stale. For some sites, a five-second window will be too large a window, and for others, too small. The sampling interval is configurable with the following registry keys, which are not present by default: Static pages Set the registry property, FileAttributeCheckThreshold, a DWORD value, located at HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Inetinfo\Parameters. ASP Scripts Set the registry property, FileMonitoringTimeoutSeconds, a DWORD value, located at HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\ASP\Parameters. Using the last-modified cache update mechanism will result in more reliable and secure delivery of content. Additionally, it will scale well on a wide application structure (where your Web server has lots of applications pointing at thousands of different directories). There is, however, a performance penalty that might be noticeable in regards to static content. Even with these new file-caching algorithms, overall performance of UNC-based content can still be slightly slower than that of local content. However, depending on the content size, the difference is usually negligible. If you think that the performance of your server has diminished due to these changes, you can enable change notificationbased caching by adding the following registry keys, which are not present by default: Static Pages Set the registry property, DoDirMonitoringForUNC, a DWORD value, located at HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Inetinfo\Parameters. ASP Scripts Set the registry property, EnableChangeNotificationForUNC, a DWORD value, located at HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\ASP\Parameters. These entries do not exist in the registry by default. To add it to the registry, use the registry editor, Regedit.exe. However, note that this option is not always recommended because it can cause the operating system to suffer from the scalability problems that were present in earlier version of IIS. Additionally, share-level permissions will no longer be honored by the Web server (NTFS ACLs are always honored). Consider modifying this registry entry only if you have a small directory structure, are not concerned about share-level permissions, and consider performance the most important metric. When deciding which caching mechanism to use, consider the following tradeoffs. The most reliable and secure method is using the default: Last-Modified-based caching. It is reliable because it does not rely on the change-notification functionality of the file server and it scales well on a wide application structure (where your Web server has lots of applications pointing at thousands of different directories). It is more secure because share-level permission changes are honored (the cache accesses the file often, which automatically triggers the authorization check against the current file system ACL and the current share ACL). However, if you are using a file server that reliably reports change notifications and if you have few sites or virtual directories, using change-notification is significantly faster. For example, if you have a single Web site with a couple of applications and all of your content is stored on a Windows Server2003 file server, it is recommended that you use the change-notification method of caching. For a complete look at using remote content on a UNC share, see the whitepaper Deploying and Configuring Internet Information Services (IIS)6.0 with Remotely Stored Content on UNC Servers and NAS Devices. You can find this article by seeing the UNC Servers Accessed by Using IIS6.0 link on the Web Resources page at http://www.microsoft.com/windows/reskits/webresources. Troubleshooting IIS Manager and UNC Content Problems Because of the way that IIS Manager displays the virtual directory, managing Web content that is stored on remote UNC shares can be confusing. IIS Manager uses the credentials of the logged-in user (the user actually running IIS Manager), who might not necessarily have the right credentials to access the remote file share. In this case, IIS Manager displays a stop-sign icon for the virtual directory. If this occurs, test UNC access by making requests with a Web browser. You might want to create virtual directories that reference UNC-mapped drives. IIS does not support mapped drives because they are user-specific. For more information, see article 257174, Using Mapped Drives with IIS in the Microsoft Knowledge Base. To find this article, see the Microsoft Knowledge Base link on the Web Resources page at http://www.microsoft.com/windows/reskits/webresources. Troubleshooting Errors That Occur When UNC Content Is Under High Load If the Web server and the remote file server are not configured properly for high stress conditions, errors might occur. The following configuration errors might be reported in the browser, in the event log on the IIS server or remote file server, or the IIS logs. Windows cannot find the network path. Verify that the network path is correct and the destination computer is not busy or turned off. If Windows still cannot find the network path, contact your network administrator. Error code: 0x80070033: The network location cannot be reached. For information about network troubleshooting, see Windows Help.Error code: 0x800704CF Not enough server storage is available to process this command. Error code: 0x8007046 Modifying the MaxUserPort Registry Entry The first two errors are returned when the server running IIS has a shortage of outbound TCP ports when it attempts to make the UNC connection. In this case, the Web server returns HTTP500 errors for all requests until TCP ports are available again. To fix this problem, you can modify the MaxUserPort registry entry on the server running IIS to specify an increase in the number of available ports. This entry does not exist in the registry by default. To add it to the registry, use the registry editor, Regedit.exe. Caution Do not edit the registry unless you have no alternative. The registry editor bypasses standard safeguards, allowing settings that can damage your system, or even require you to reinstall Windows. If you must edit the registry, back it up first and see the Registry Reference on the Microsoft Windows Server2003 Deployment Kit companion CD or on the Web at HYPERLINK "http://www.microsoft.com/reskit" http://www.microsoft.com/reskit. For example, if you have many concurrent authenticated users, a different SMB connection is opened from the computer running IIS to the remote file server for each individual user. These SMB connections use ports. By default, you are limited to ports from 1024 to 5000 for outbound TCP connections (slightly fewer than 4,000 ports are available), so you might need to increase this value to a higher number, such as 10000 or 20000, to provide additional ports for SMB connections. Be aware that this might cause problems with applications that create sockets on these higher ports. For more information, see Microsoft Windows2000 TCP/IP Implementation Details. Checking the File Server Setting If you receive the Not enough server storage is available to process this command. Error code: 0x8007046A message, check your file server settings. This IIS error usually accompanies event log messages like the following: The server was unable to allocate from the system nonpaged pool because the pool was empty. The server was unable to allocate a work item