advanced windows debugging
Post on 24-Dec-2014
670 Views
Preview:
DESCRIPTION
TRANSCRIPT
Advanced Windows Debugging
Mario HewardtSenior Development LeadMicrosoft Corporation
Daniel PravatSenior Development LeadMicrosoft Corporation
…and we approve this message
Software Bugs Are EXPENSIVE
A $60 billion problem “Software bugs, or errors, are so prevalent
and so detrimental that they cost the U.S. economy an estimated $59.5 billion annually”
Users choose reliability over price Reliability problems are sticky
Why Debugging?We Need Reliable Software
Agenda
10:15 – 11:30 Introduction to Debuggers
11:30 – 12:00 Managing Symbols/Sources
12:00 – 1:00 Lunch1:00 – 1:55 Resource Leaks2:00 - 3:00 Security3:15 - 3:45 Break3:45 - 4:40 Heap Corruptions4:45 - 5:15 Windows Error Reporting5:15 - 5:45 Managed Code Debugging
Session Structure
Session breaks Q&A feedback loop
AWD PDC 2008 Feedback/Questions Focus is on user mode & native code
debugging Sessions emphasize pragmatism over
theory Advanced Windows Debugging book
coupons (AW Booth 517) Special guests
Example of bugs repartition Open source code
Source: Coverity White Paper
Introduction to the Debugging Tools with examples The basics about the Debugging Tools for
Windows How to set them up How to work with symbols and sources Basic commands available in the Debugging
Tools for Windows Several debugging scenarios How to use the Windows debuggers
remotely
Debugging Tools for Windows
xcopy “installable” Always up to date Low memory footprint Extensible Windows experience build it Public version (search Windbg on the web) http://www.microsoft.com/whdc/devtools/debug
ging/default.mspx Public version update 2 times a year Used everywhere (including ARM devices) Inexpensive high quality tools
User mode debuggers Cdb.exe Ntsd.exe Windbg.exe
Kernel mode debuggers Kd.exe Windbg.exe
Symbol indexing tools Source indexing tools Stand-alone tools (agestore, adplus, breakin,
dbgsrv, gflags, tlist, remote, etc. ) Samples
Package Content
Debugger selection Live debugging vs. crash mode debugging User mode vs. kernel mode Start the UM application under the
debugger or attach to a running instance windbg.exe <app_to_start> windbg –p <PID> windbg –pn <exe_name> windbg –z <crash_dump>
Debugger interaction Multiple debuggers to the same target
Starting the Debugger
Command mode vs. GUI User mode prompt can get us a head start The first step is getting the exception code Understand the environment Set the correct symbols Start from the current execution context Check the loaded module
Interacting with the Debugger
Simple commands02sample.exe Daniel Pravat
demo
Symbol types PDB information Setting the symbol path Checking the symbol loading process Using a symbol server (private vs. public) Using a symbol cache Maintaining the symbol cache
Symbol File & Module Information
Setting the environment Setting the symbols Loading the symbols Loading the source files
Inspecting the target Immutable, read the information from kernel and from the
process space Last event Registers Memory Variables Stack Unassemble Process information Thread information Address information
Working with the Target
Sample commands02sample.exe Daniel Pravat
demo
Breakpoints Breakpoint types Breakpoint info Setting a breakpoint (code BP, processor BP) Conditional breakpoint
Controlling the target Execute Trace
Debugger events What are they? Useful events Controlling the events
Helper commands Math Errors Shell execute
Breakpoints, Events and Others
Sample commands02sample.exe Daniel Pravat
demo
Deadlock10deadlock.exe Daniel Pravat
demo
cs_DB1cs_DB1
cs_DB2
Deadlock Scenario
cs_DB2
Thread 0
Thread 1
No progress = Deadlock
Acquisition order is reversedThe length may be different A,B,C = OKB,C = OKA,B = OKA,C = OKB,C,A = Deadlock
What is it? When to use it? Remote debugging requirements Client command <debugger> –remote <protocol>:<protocol options>
Symbol resolution in remote debugging Source resolution Debug server Non-traditional use in image file execution
options.
Remote Debugging
Offer debugging capabilities for remote labs Small bandwidths Allows testing with limited number of tools
installed
Remote Debugging
Test System
Test System
Test System
Test System
Test Systems
Remote client
SymbolsSources
Smart client
Sources
Stack corruption05async.exe Daniel Pravat
demo
Possible Explanation for Failure09async.exe
DisplayError calls Sleep
Return address is saved
Sleep calls SleepEx
Return address is saved
…
Return address is changed to X
Execution starts at X
There is another thread smashing the stack
How to catch this async operation?
What are the implications of this bug?
Can the execution be controlled? Can this execution be prevented? Welcome NX bit
Tools are free, easy to find, easy to use Use every bit from the debugger target state to
understand the mechanics of the crash Build an hypothesis that can explain the current
problem Reproduce (mentally is fine) the current problem.
If you can’t, build another hypothesis. Find the root cause in the source file Debugging doesn’t stop at component boundary Use remote debugging when you need concurrent
access to the same target
Summary
Questions
Symbol server
Daniel Pravat
Build a new
version
Extract public
symbols
Index symbols
Publish symbols
Use symbols
What is a symbol server? When in the best time to do it? What is the cost-benefit ratio?
Maintaining a Symbol Server
Large store of symbol and binary files Files are organized based on unique
properties of each file: File name File type Time stamp Size of the image RSDS signature
The binary files can be stored on a different location
The files can be compressed
Symbol Server
There is too much information in the symbol files
Small companies can manage the symbols Symbols are large ($.10/GB) We ship infrequently We have no bugs, we don’t debug We use tracing We ship often The code does not change, I can rebuild PDBs are added to the product
Common Misconceptions
Indexing step by step Collecting all binary files in a single location Generate equivalent public symbol files Store them in the final location if not stored on
the symbol server Add both sets of files to their symbol servers
Tools required Pdbcopy.exe (Debugging Tools for Windows) Binplace.exe (WDK) Symstore.exe (Debugging Tools for Windows) Extending the build process (batch files)
Can be done
Building a Symbol Server
Use them from a share (private symbols) Publish the symbols on a HTTP server
(public symbols) Publish them on a secure HTTP server
(private symbols) Purging intermediate build versions The most cost effective way of securely
providing the correct symbols Essential for library vendors
Using the Symbol Server
A problem ~= a defect in the source file The source file should be at exact version Windbg.exe can use source server .srcpath srv*
Source Server
Build a new
version
Extract the source
file list
Store the information required to retrieve the file from SC
Publish symbols
Use the symbols
It is a concept Private symbol PDBs contain the original
file location Source server is an alternate stream into
the private symbol PDBs Contains information to retrieve the source
file from the source revision control system Independent of the source revision control
system Works with: Perforce, Visual SourceSafe,
Team Foundation Server, Subversion
Source Server
Source Server Information (VSS)
C:\>pdbstr –r –p:%_NT386TREE%\sym.pri\retail\exe\03sample.pdb –s:srcsrvSRCSRV: ini ------------------------VERSION=1INDEXVERSION=2VERCTRL=Visual Source SafeDATETIME=Mon Jan 8 00:04:15 2007SRCSRV: variables ---------------------SSDIR=C:\AWD\VSSSRCSRVENV=SSDIR=%AWD%VSSTRGDIR=%targ%\%var2%\%fnbksl%(%var3%)\%var4%VSS_EXTRACT_CMD=ss.exe get -GL”%vsstrgdir%” -GF- -I-Y -W “$/%var3%” -V”%var4%”VSS_EXTRACT_TARGET=%targ%\%var2%\%fnbksl%(%var3%)\%var4%\%fnfile%(%var1%)AWD=C:\AWD\VSSSRCSRVTRG=%VSS_extract_target%SRCSRVCMD=%VSS_extract_cmd%SRCSRV: source files --------------------c:\awd\chapter3\spydbg.cpp*AWD*chapter3/spydbg.cpp*VERSION1SRCSRV: end ------------------------
Have customers, manage symbols Symbol server is cost effective One place to manage them all Allow private - publish separation Works with other tools (Process Monitor,
Profilers) Source server helps localize the bugs Can be integrated in most processes Request support from your toolset vendor
Summary
Questions
Resource Leaks
What is a Resource?
Anything that occupies space in the system Simple right?
Resources come in many forms Handles Synchronization primitives Heap memory allocator Virtual memory allocator and many more…
Don’t let the abstraction fool you -- it all boils down to memory
The means by which a resource is acquired and released is key
What is a Resource Leak?
Owning process acquires a resource but fails to release it hHandle = CreateEvent (…,…); QueryInterface(…,…, &pI); pMem=HeapAlloc(…,…);
Not confined within the boundaries of a process
It’s evil. Especially when not easily reproduced
Why are resource leaks bad?
Resources are limited Someone else has to give
Availability goes down Performance degradation Just like any other bug, cost to fix is high
after release Drives the hardware market
Top resource leaks misconceptions
Windows cleans it up so I don’t have to I’ll just recycle if it gets out of hand The code I was calling was supposed to
clean it up!
Resource leaks symptoms
Quantity of a given resource keeps increasing over time Handle count goes up Virtual memory usage goes up Be wary of caching techniques
System becomes sluggish Sporadic failures Inability to start new applications Processes flat out dies
Types of Resources
Handles Heap allocator Virtual memory allocator COM allocator and many more…
Handles
User mode identifier of Windows object types File object Process object Thread object
Applications use the handle to interact with OS objects
Best viewed as an isolation layer between user mode code and the kernel
Handle Architecture Overview
EPROCESS
Ref Count Obj Count Object1 1 <addr>1 1 <addr>3 1 <addr>
Event
WIN32 API
MutantHeader
Header
CreateEvent(…
…);
UserKernel
Handle Code Example
HANDLE hFile=CreateFile( pWorkerData->pszFileName, FILE_READ_DATA,
FILE_SHARE_READ, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
// Use file handle
CloseHandle(hFile);
Tools for Handle Tracking
Debugger extension command: !htrace Task Manager
Shows handle count Process Explorer shows handle count as
well as: Type of handle (file, mutant, section etc) Name of the handle Handle value Comes in handy when figuring out what type of
handle is being leaked http://technet.microsoft.com/en-us/
sysinternals/bb896653.aspx
Resolution process
Leak?
Type ofresource
InitialAnalysis
Use tools
AvoidanceStrategy
DoneNo
Basic Handle Leak
Mario Hewardt
Demo
Inconsistent Handle Leak
Mario Hewardt
Demo
Cross Process Handle Leak
Mario Hewardt
Demo
Heap Memory
Most commonly used allocator for user mode applications
Uses the virtual memory manager Provides more manageable chunks of
memory than virtual memory manager Other allocators are built on top of heap
manager
Windows Memory Architecture
Application
Virtual Memory Manager
Heap Manager
Default Process
Heap
C RuntimeHeap Other Heaps
Heap Memory Code Example
BYTE* pMem=(BYTE*) HeapAlloc(GetProcessHeap(), 0, 100);
// Use memory
HeapFree(GetProcessHeap(), 0, pMem);
Tools for Heap Memory Tracking
UMDH Tracks heap based memory Requires OS instrumentation to be enabled
(gflags) LeakDiag
Tracks different types of memory allocators: Heap allocator Virtual memory allocator COM allocator C runtime allocator
Uses Microsoft Detours library Debugger command: !heap
Memory Leak using UMDH
Mario Hewardt
Demo
Memory Leak using LeakDiag
Mario Hewardt
Demo
Preemptive Strategies
Static source code analysis tools Prefast (available with WDK)
Always run with UMDH or LeakDiag At the very least during select milestones
Memory wrappers
Questions
DebuggingSecurity
Agenda
The basics of Windows security Inspecting security elements
Debugging a distributed scenario Debugging a simple in process scenario Alternative way to debug it
Past: Lack security Current generation: C2 compliant OS
(Starting with Windows 3.5 SP3) Most of the users in unmanaged
environment are administrators on the system
The security related problems are visible when running as a normal privilege user
UAC introduced in Windows Vista Vista and Windows Server 2008 uses
service accounts
Security is Important
Complex systems are harder to debug The complexity must be understood The failure origin is often lost There are many layers participating Each layer can make a security check Upper layer sees generic errors
Complex System Use Security Mechanisms
Web client Web front-end Middle tier Database
back-end
By probing the execution path Each component offers a probing point Most probing points are provided by the
operating system On the probing point we can identify
The principal making the operation Who has right for that operation If the execution path reaches that component
Empiric process Complexity reduction
Debugging Complex Systems
Autorization model
AccessCheck Allowed Denied
Authorization ComponentsThe User Identity (principal)
The equivalent of a passport There is a 3rd trusted issuer Contains user membership and other claims It is a secure container, its content cannot
be falsified It is used for all operations performed
against a protected object Each process started on the system
operates on behalf of the user !token can decipher the user token
Displaying an access token
0:000> !token 7bc -nTS Session ID: 0User: S-1-5-21-1060284298-2111687655-1957994488-1003 (User: XP-SP2\TestAdmin)Groups:00 S-1-5-21-1060284298-2111687655-1957994488-513 (Group: XP-SP2\None)Attributes - Mandatory Default Enabled01 S-1-1-0 (Well Known Group: localhost\Everyone)Attributes - Mandatory Default Enabled02 S-1-5-32-544 (Alias: BUILTIN\Administrators)Attributes - Mandatory Default Enabled Owner03 S-1-5-32-545 (Alias: BUILTIN\Users)Attributes - Mandatory Default Enabled04 S-1-5-4 (Well Known Group: NT AUTHORITY\INTERACTIVE)Attributes - Mandatory Default Enabled05 S-1-5-11 (Well Known Group: NT AUTHORITY\Authenticated Users)Attributes - Mandatory Default Enabled06 S-1-5-5-0-35778 (no name mapped)Attributes - Mandatory Default Enabled LogonId07 S-1-2-0 (Well Known Group: localhost\LOCAL)Attributes - Mandatory Default EnabledPrimary Group: S-1-5-21-1060284298-2111687655-1957994488-513 (Group: XP-SP2\None)Privs:00 0x000000017 SeChangeNotifyPrivilege Attributes - Enabled Default01 0x000000008 SeSecurityPrivilege Attributes -...17 0x000000009 SeTakeOwnershipPrivilege Attributes -18 0x00000001e SeCreateGlobalPrivilege Attributes - Enabled Default19 0x00000001d SeImpersonatePrivilege Attributes - Enabled DefaultAuth ID: 0:1c3a8Impersonation Level: IdentificationTokenType: Impersonation
The equivalent of border regulation Many objects, many rules A collection of rights allowed to different
identities Each object has a security descriptor that
can allow or deny access to the object Granted access cannot be taken away DACLs are not accessible to the user of the
object !dacl, !sid, !acl extension can by used in
debugger
Authorization ComponentsThe Discretionary Access Control List (DACL)
Displaying ACLs
0:000> !acl 000840acACL is:ACL is: ->AclRevision: 0x2ACL is: ->Sbz1 : 0x0ACL is: ->AclSize : 0x1cACL is: ->AceCount : 0x1ACL is: ->Sbz2 : 0x0ACL is: ->Ace[0]: ->AceType: ACCESS_ALLOWED_ACE_TYPEACL is: ->Ace[0]: ->AceFlags: 0x0ACL is: ->Ace[0]: ->AceSize: 0x14ACL is: ->Ace[0]: ->Mask : 0x00120089ACL is: ->Ace[0]: ->SID: S-1-1-0
Displaying Security Descriptors
kd> !sd 00084098->Revision: 0x1->Sbz1 : 0x0->Control : 0x8004SE_DACL_PRESENTSE_SELF_RELATIVE->Owner : S-1-5-18->Group : S-1-5-32-544->Dacl :->Dacl : ->AclRevision: 0x2->Dacl : ->Sbz1 : 0x0->Dacl : ->AclSize : 0x1c->Dacl : ->AceCount : 0x1->Dacl : ->Sbz2 : 0x0->Dacl : ->Ace[0]: ->AceType: ACCESS_ALLOWED_ACE_TYPE->Dacl : ->Ace[0]: ->AceFlags: 0x0->Dacl : ->Ace[0]: ->AceSize: 0x14->Dacl : ->Ace[0]: ->Mask : 0x00120089->Dacl : ->Ace[0]: ->SID: S-1-1-0->Sacl : is NULL
The equivalent of border policeEnforces the DACLs accessCan be a custom SRM
Implemented as a different process DACL information is protected The resources must be protected Example: DCOM
Lives in a different security boundaryDemo
Authorization ComponentsThe Security Reference Monitor (SRM)
!token wmiprvse.exe Daniel Pravat
demo
DCOM Example
void MTAClientCall(){ COSERVERINFO si;
MULTI_QI mqi; ... if (SUCCEEDED(CoCreateInstanceEx( CLSID_Calculator, NULL, CLSCTX_LOCAL_SERVER | CLSCTX_REMOTE_SERVER, &si, 1, &mqi ))) { ICalculator * pCalculator = NULL; pCalculator = (ICalculator*)mqi.pItf; __int32 result = 0; wprintf(L"Calling SumSlow ..."); pCalculator->SumSlow(1,2, &result); pCalculator->Release(); wprintf(L"done\n"); }}
DCOM CoCreateInstance complexity
Not all connections are shown Each box has IPC code Each box performs security validation The client cannot identify the failing
operation
Remote client
RPCSS Server
DCOM server
DCOM LaunchHOST OS SCM
DCOM client
08cli.exe
Daniel Pravat
demo
Summary
Security, usability or both? Security can be a differentiator Secure by default in the new wave Enable security after understanding it Requires good knowledge of the underlying
components and the ability to step into such systems
Debugging by probing The landscape changes, be prepared!
Questions
Heap Corruptions
What is a Heap Corruption?
Violate the integrity of memory allocated on the heap Stray pointers Overruns Underruns Over-deletion Re-use after deletion Many others
Arguably one of the toughest categories of problems to debug
Common Misconceptions
Top most frame not in my code!000bfdf4 7781f676 ntdll!RtlpCoalesceFreeBlocks+0x4f0
000bfeec 7781f285 ntdll!RtlpFreeHeap+0x1e2
000bff08 75f93593 ntdll!RtlFreeHeap+0x14e
000bff1c 010012cc kernel32!HeapFree+0x14
000bff38 01001302 06overrun+0x12cc
000bff44 0100147c 06overrun+0x1302
000bff88 75f9e3f3 06overrun+0x147c
…
…
…
Common Misconceptions (cont)
My code is not on the stack period! It must be a hardware problem! I use my own heap, cant possibly corrupt
someone elses’ heap Window of opportunity is too small. It will
never happen
Heap Corruption Symptoms
Wide range of strange behavior can be exhibited
Application crashes Application hangs Application exhibits “undesirable” behavior. If you are lucky, symptom close to source More commonly, symptom far from source
Windows Memory Architecture
Application
Virtual Memory Manager
Heap Manager
Default Process
Heap
C RuntimeHeap Other Heaps
Heap Manager: Frontend Allocator
Look aside Table
0
1
2
3
…
127
Unused
16
24
32
…1024
Heap Manager: Backend Allocator
0
1
2
3
…
127
Variable Size
unused
16
24
…1016
Free Lists12
X…
Segment List
Heap Segments
Or where does the memory come from? One ore more chunks of virtual memory Each segment contains heap blocks Structural integrity maintained by the heap
manager Represented by _HEAP_SEGMENT New segments created as needed
Heap Segment Structure
Busy Block Busy Block Uncommitted rangeFree Block
Heap Blocks
Basic allocation unit Contained within segment Contains allocation metadata
Allows heap manager to effectively manage the memory
Can contain debug info depending on what instrumentation is enabled
Heap Block Structure
Current Size
Previous Size
SegIndex Flags Unused Tag
Index
Pre-allocation Metadata Post-allocation Metadata
User accessible part
Pre-allocation Metadata
Suffix Bytes
Fill Area(debug)
Heap Extra
Post-allocation Metadata
User accessible part
User accessible part
Heap Coalescing
A mechanism to avoid fragmentation Avoids memory failures when not enough
contiguous memory is available Merges adjacent free blocks into large block Imagine freeing an allocation of size 32
Allocation Size: 32Allocation Size: 16 Allocation Size: 16
Allocation Size: 64
Tools for Debugging Heap Corruptions
Goal is to break when the corruption occurs and not after
Pageheap helps with that goal Annotates heap blocks to trigger fault at the
time of write Light pageheap uses fill patterns Full pageheap uses fill patterns and guard
pages Very memory intensive
Heap Overrun
Mario Hewardt
Demo
Reuse after Deletion
Mario Hewardt
Demo
Heap Handle Mismatch
Mario Hewardt
Demo
Questions
Windows Error Reporting
Windows Error Reporting (WER)
Allows you to track the health of your applications in the wild
Failure data aggregation service provided by Microsoft
Signup is easy and free Certificate required for verification purposes VeriSign organizational Certificate
https://winqual.microsoft.com/
WER Functionality
Allows ISVs to Manage applications to be monitored Query for application fault data Retrieve extensive application fault data Provide a feedback loop for customers Also allows hardware fault monitoring
Application Fault Data
Downloaded as CAB files Each CAB file contains fault data
Perhaps most important is the dump file ISV’s can customize the type of dump file to
collect ISV’s can customize additional data to
collect Additional data files can be collected Extensive machine information
WER Architecture
Error SentDr.
WatsonProcessCrash
Crash data over HTTPS
Fault response over HTTPS
Windows Error Reporting Service
ISV
Query Fault Data
Windows Error Reporting
Mario Hewardt
Demo
Programmatic Access to WER
WER exposed as a set of Web Services Open source project on codeplex
http://www.codeplex.com/wer/ Initial projects include
Client DLL Sample web service interaction Gadget to track events
Develop customized WER applications for you company
WER Summary
Great way to gauge the health of your application
Its free! Extensive fault data collection policies Provides the ability to close the feedback
loop with customers
Questions
Managed code supportDaniel Pravat
Minimal support through the sos.dll debugger extension Shipped with each framework version Easy to find it Well documented
Minimal integration with the debugger The integration point may increase in time No good understanding of the market What does not work
Variable/type dump Breakpoints
Managed Code Support in Debugging Tools for Windows
Crash mode Over-stressed systems
Other tools may not work Mixing with native components No sources available Debugging dump files Debugging interop (Pinvoke) layers Debugging on customer site Debugging retail code Need to use tools designed for native code Kernel mode redirection Nothing else works
When to Use Debugging Tools for Windows
During code development Tracing the code 100% managed code Need frequent variable inspection Need frequent references to the source files Debugging partial dumps Kernel mode debugging
SOS needs access to the entire address space Some pages are paged out
When Not to Use Debugging Tools for Windows
Sample Code
Main(){... int operationCount = 0; for (; ; ) { operationCount++; HostInfo hi = HostInfo.GetDefault(hostAddress, userName, password); IWSManSession session = GetSession(hi); string response = session.Get("wmicimv2/Win32_OperatingSystem", 0); Console.Write("\rNumber of calls: " + operationCount); }...}
Sample code (cont.)
static public IWSManSession GetSession(HostInfo hi) { IWSManSession session = null; // Get a cached session string key = hi.GetKey(); if (!sessionCache.TryGetValue(key, out session)) { session = CreateSession(hi); sessionCache[key] = session; } return session; }
Monitor.exe
Daniel Pravat
demo
Summary
Managed code applications are partially native
Use the best tools for the job Sometime the best tools were designed for
native processes The knowledge is portable to the managed
world Working knowledge of CLR internals
Additional Support in Debugging Tools for Windows
Increased CLR presence Reliability focus Now we have minimal integration
JIT compilation Requires information from the process, not
from PDBs Future (wish list)
Each operation intercepted by CLR support All commands work (dv works!) Seamless transition from native to managed
Everyone loves reliable applications Be aware of magic The code is just that, code Be curious and adventurous, explore new
areas Patience is key Use the tools extensively THANK YOU !
Summary
Books: Advanced Windows Debugging
by Mario Hewardt, Daniel PravatAddison-Wesley Professional (November 8, 2007)Free chapter on book’s site
Windows® Internals: Including Windows Server 2008 and Windows Vista, Fifth Edition, by Mark E. Russinovich and David A. Solomon
Windows via C/C++ by Jeffrey M. Richter; Christophe Nasarre Microsoft Press; 5th edition (December 12, 2007)
Blogs: Microsoft Advanced Windows Debugging and Troubleshooting Crash Dump Analysis If broken it is, fix it you should
Usenet Microsoft.public.windbg
Refferences
TL59 – VS Debugger Tips and Tricks Driving for software quality through
customer feedback (web exclusive) Windows Fundamentals Kiosk Addison Wesley Kiosk
Don’t Miss
Questions
© 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market
conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
top related