Windows Operating System Internals - by David A. Solomon and Mark E. Russinovich with Andreas PolzeWindows Operating System Internals - by David A. Solomon and Mark E. Russinovich with Andreas Polze

Unit OS6: Device ManagementUnit OS6: Device Management

6.3.6.3. Windows I/O ProcessingWindows I/O Processing

2

Copyright NoticeCopyright Notice© 2000-2005 David A. Solomon and Mark Russinovich© 2000-2005 David A. Solomon and Mark Russinovich

These materials are part of the These materials are part of the Windows Operating Windows Operating System Internals Curriculum Development Kit,System Internals Curriculum Development Kit, developed by David A. Solomon and Mark E. developed by David A. Solomon and Mark E. Russinovich with Andreas PolzeRussinovich with Andreas Polze

Microsoft has licensed these materials from David Microsoft has licensed these materials from David Solomon Expert Seminars, Inc. for distribution to Solomon Expert Seminars, Inc. for distribution to academic organizations solely for use in academic academic organizations solely for use in academic environments (and not for commercial use)environments (and not for commercial use)

3

Roadmap for Section 6.3Roadmap for Section 6.3

Driver and Device ObjectsDriver and Device Objects

I/O Request Packets (IRP) ProcessingI/O Request Packets (IRP) Processing

Driver Layering and FilteringDriver Layering and Filtering

Plug-and-Play (PnP) and Power Manager Plug-and-Play (PnP) and Power Manager OperationOperation

Monitoring I/O Activity with FilemonMonitoring I/O Activity with Filemon

4

Driver ObjectDriver Object

A driver object represents a loaded driverA driver object represents a loaded driver

Names are visible in the Object Manager Names are visible in the Object Manager namespace under \Driversnamespace under \Drivers

A driver fills in its driver object with pointers to its I/O A driver fills in its driver object with pointers to its I/O functions e.g. open, read, writefunctions e.g. open, read, write

When you get the “One or More Drivers Failed to When you get the “One or More Drivers Failed to Start” message its because the Service Control Start” message its because the Service Control Manager didn’t find one or more driver objects in the Manager didn’t find one or more driver objects in the \Drivers directory for drivers that \Drivers directory for drivers that shouldshould have have startedstarted

5

Device ObjectsDevice Objects

A device object represents an instance of a A device object represents an instance of a devicedevice

Device objects are linked in a list off the driver Device objects are linked in a list off the driver objectobject

A driver creates device objects to represent the A driver creates device objects to represent the interface to the logical device, so each generally has interface to the logical device, so each generally has a unique name visible under \Devicesa unique name visible under \Devices

Device objects point back at the Driver objectDevice objects point back at the Driver object

6

Driver and Device ObjectsDriver and Device Objects

\TCPIP

Driver Object \Device\TCP \Device\UDP \Device\IP

Dispatch Table

Open

Write

Read

Loaded Driver Image

Open(…)

Read(…)

Write(…)

TCP/IP Drivers Driver and Device Objects

7

File ObjectsFile Objects

Represents open instance of a device (files on a volume are virtual Represents open instance of a device (files on a volume are virtual devices)devices)

Applications and drivers “open” devices by nameApplications and drivers “open” devices by name

The name is parsed by the Object ManagerThe name is parsed by the Object Manager

When an open succeeds the object manager creates a file object to When an open succeeds the object manager creates a file object to represent the open instance of the device and a file handle in the represent the open instance of the device and a file handle in the process handle tableprocess handle table

A file object links to the device object of the “device” which is A file object links to the device object of the “device” which is openedopened

File objects store additional informationFile objects store additional information

File offset for sequential accessFile offset for sequential access

File open characteristics (e.g. delete-on-close)File open characteristics (e.g. delete-on-close)

File nameFile name

Accesses granted for convenienceAccesses granted for convenience

8

I/O Request PacketsI/O Request Packets

System services and drivers allocate I/O request packets to describe I/OSystem services and drivers allocate I/O request packets to describe I/O

A request packet contains:A request packet contains:

File object at which I/O is directedFile object at which I/O is directed

I/O characteristics (e.g. synchronous, non-buffered)I/O characteristics (e.g. synchronous, non-buffered)

Byte offsetByte offset

LengthLength

Buffer locationBuffer location

The I/O Manager locates the driver to which to hand the IRP by following The I/O Manager locates the driver to which to hand the IRP by following the links:the links:

File Object Device Object Driver Object

9

Putting it Together: Request FlowPutting it Together: Request Flow

Process

NtDeviceIoControlFile

Handle

Table

File

Object

Device

Object

Driver

Object

Dispatch

Table

DispatchDeviceControl( DeviceObject, Irp )

Driver Code

User Mode

Kernel Mode

IRP

DeviceIoControl

10

I/O Request PacketI/O Request Packet

Environmentsubsystem or

DLL

Services

I/O manager

IRP header

WRITEparameters

Fileobject

Deviceobject

Driverobject

IRP stacklocation

Dispatchroutine(s) Start I/O ISR

DPC routine

Device Driver

1)An application writesa file to the printer,passing a handle tothe file object

2)The I/O manager creates an IRP and initializes first stack location

3)The I/O manager usesthe driver object to locatethe WRITE dispatch routine and calls it, passing the IRP

User modeKernel mode

11

IRP dataIRP data

IRP consists of two parts:IRP consists of two parts:

Fixed portion (header):Fixed portion (header):

Type and size of the requestType and size of the request

Whether request is synchronous or asynchronousWhether request is synchronous or asynchronous

Pointer to buffer for buffered I/OPointer to buffer for buffered I/O

State information (changes with progress of the request)State information (changes with progress of the request)

One or more stack locations:One or more stack locations:

Function codeFunction code

Function-specific parametersFunction-specific parameters

Pointer to caller‘s file objectPointer to caller‘s file object

While active, IRPs are stored in a thread-specific queueWhile active, IRPs are stored in a thread-specific queue

I/O system may free any outstanding IRPs if thread terminatesI/O system may free any outstanding IRPs if thread terminates

12

I/O Processing –I/O Processing –synch. I/O to a single-layered driversynch. I/O to a single-layered driver

1.1. The I/O request passes through a subsystem DLLThe I/O request passes through a subsystem DLL

2.2. The subsystem DLL calls the I/O manager‘s NtWriteFileThe subsystem DLL calls the I/O manager‘s NtWriteFile()() service service

3.3. I/O manager sends the request in form of an IRP to the driver (a device I/O manager sends the request in form of an IRP to the driver (a device driver)driver)

4.4. The driver starts the I/O operationThe driver starts the I/O operation

5.5. When the device completes the operation and interrupts the CPU, the When the device completes the operation and interrupts the CPU, the device driver services the int.device driver services the int.

6.6. The I/O manager completes the I/O requestThe I/O manager completes the I/O request

13

Completing an I/O requestCompleting an I/O request

Servicing an interrupt:Servicing an interrupt:

ISR schedules Deferred Procedure Call (ISR schedules Deferred Procedure Call (DPCDPC); dismisses int.); dismisses int.

DPCDPC routine starts next I/O request and completes interrupt servicing routine starts next I/O request and completes interrupt servicing

May call completion routine of higher-level driverMay call completion routine of higher-level driver

I/O completion:I/O completion:

Record the outcome of the operation in an I/O status blockRecord the outcome of the operation in an I/O status block

Return data to the calling thread – by queuing a kernel-mode Return data to the calling thread – by queuing a kernel-mode Asynchronous Procedure Call (Asynchronous Procedure Call (APCAPC))

APCAPC executes in context of calling thread; copies data; frees IRP; executes in context of calling thread; copies data; frees IRP;sets calling thread to signaled statesets calling thread to signaled state

I/O is now considered complete; waiting threads are releasedI/O is now considered complete; waiting threads are released

14

Flow of InterruptsFlow of Interrupts

Peripheral Device Controller

CPU Interrupt Controller

CPU Interrupt Service Table

0

2

3

n

ISR Address

Spin Lock

Dispatch Code

Interrupt Object

Read from device

Acknowledge-Interrupt

Request DPC

Driver ISR

Raise IRQL

Lower IRQL

KiInterruptDispatch

Grab Spinlock

Drop Spinlock

15

Servicing an Interrupt:Servicing an Interrupt:Deferred Procedure Calls (DPCs)Deferred Procedure Calls (DPCs)

Used to defer processing from higher (device) interrupt level to a lower Used to defer processing from higher (device) interrupt level to a lower (dispatch) level(dispatch) level

Also used for quantum end and timer expirationAlso used for quantum end and timer expiration

Driver (usually ISR) queues requestDriver (usually ISR) queues request

One queue per CPU. DPCs are normally queued to the current processor, but One queue per CPU. DPCs are normally queued to the current processor, but can be targeted to other CPUscan be targeted to other CPUs

Executes specified procedure at dispatch IRQL (or “dispatch level”, also “DPC Executes specified procedure at dispatch IRQL (or “dispatch level”, also “DPC level”) when all higher-IRQL work (interrupts) completedlevel”) when all higher-IRQL work (interrupts) completed

Maximum times recommended: ISR: 10 usec, DPC: 25 usecMaximum times recommended: ISR: 10 usec, DPC: 25 usec

See See http://www.microsoft.com/http://www.microsoft.com/whdc/driver/perform/mmdrv.mspxwhdc/driver/perform/mmdrv.mspx

queue head DPC object DPC object DPC object

16

DPC

Delivering a DPCDelivering a DPC

DPC routines can call kernel functionsbut can‘t call system services, generatepage faults, or create or wait on objects

DPC routines can‘tassume whatprocess addressspace is currentlymapped

Interruptdispatch table

high

Power failure

Dispatch/DPC

APC

Low

DPC

1. Timer expires, kernelqueues DPC that willrelease all waiting threadsKernel requests SW int.

DPCDPC

DPC queue

2. DPC interrupt occurswhen IRQL drops belowdispatch/DPC level

dispatcher

3. After DPC interrupt,control transfers tothread dispatcher

4. Dispatcher executes each DPCroutine in DPC queue

17

I/O Completion: I/O Completion: Asynchronous Procedure Calls (APCs)Asynchronous Procedure Calls (APCs)

Execute code in context of a particular user threadExecute code in context of a particular user thread

APC routines can acquire resources (objects), incur page faults,APC routines can acquire resources (objects), incur page faults,call system servicescall system services

APC queue is thread-specificAPC queue is thread-specific

User mode & kernel mode APCsUser mode & kernel mode APCs

Permission required for user mode APCsPermission required for user mode APCs

Executive uses APCs to complete work in thread spaceExecutive uses APCs to complete work in thread space

Wait for asynchronous I/O operationWait for asynchronous I/O operation

Emulate delivery of POSIX signalsEmulate delivery of POSIX signals

Make threads suspend/terminate itself (env. subsystems)Make threads suspend/terminate itself (env. subsystems)

APCs are delivered when thread is in alertable wait stateAPCs are delivered when thread is in alertable wait state

WaitForMultipleObjectsEx(), SleepEx()WaitForMultipleObjectsEx(), SleepEx()

18

Asynchronous Procedure CallsAsynchronous Procedure Calls(APCs)(APCs)

Special kernel APCsSpecial kernel APCs

Run in kernel mode, at IRQL 1Run in kernel mode, at IRQL 1

Always deliverable unless thread is already at IRQL 1 or aboveAlways deliverable unless thread is already at IRQL 1 or above

Used for I/O completion reporting from “arbitrary thread context”Used for I/O completion reporting from “arbitrary thread context”

Kernel-mode interface is linkable, but not documentedKernel-mode interface is linkable, but not documented

““Ordinary” kernel APCsOrdinary” kernel APCs

Always deliverable if at IRQL 0, unless explicitly disabled Always deliverable if at IRQL 0, unless explicitly disabled (disable with KeEnterCriticalRegion)(disable with KeEnterCriticalRegion)

User mode APCsUser mode APCs

Used for I/O completion callback routines (see ReadFileEx, WriteFileEx); also, Used for I/O completion callback routines (see ReadFileEx, WriteFileEx); also, QueueUserApcQueueUserApc

Only deliverable when thread is in “alertable wait”Only deliverable when thread is in “alertable wait”

ThreadObject

K

U

APC objects

19

Driver Layering and FilteringDriver Layering and Filtering

To divide functionality across To divide functionality across drivers, provide added value, etc. drivers, provide added value, etc.

Only the lowest layer talks to Only the lowest layer talks to the I/O hardwarethe I/O hardware

““Filter drivers” attach their Filter drivers” attach their devices to other devicesdevices to other devices

They see all requests first and They see all requests first and can manipulate themcan manipulate them

Example filter drivers:Example filter drivers:

File system filter driverFile system filter driver

Bus filter driverBus filter driver

ProcessProcess

User Mode

Kernel Mode

System ServicesSystem Services

I/O ManagerI/O Manager

File System Driver

File System Driver

Volume Manager

Driver

Volume Manager

Driver

Disk DriverDisk Driver

IRPIRP

20

Driver Filtering: Driver Filtering: Volume Shadow CopyVolume Shadow Copy

New to XP/Server 2003New to XP/Server 2003

Addresses the “backup open files” problemAddresses the “backup open files” problem

Volumes can be “snapshotted”Volumes can be “snapshotted”

Allows “hot backup” (including open files)Allows “hot backup” (including open files)

Applications can tie in with mechanism to ensure consistent snapshotsApplications can tie in with mechanism to ensure consistent snapshots

Database servers flush transactionsDatabase servers flush transactions

Windows components Windows components such as the Registry such as the Registry flush data filesflush data files

Different snapshot providers Different snapshot providers can implement different snapshot can implement different snapshot mechanisms:mechanisms:

Copy-on-writeCopy-on-write

MirroringMirroring

Volsnap is the built-in provider:

•Built into Windows XP/Server 2003

•Implements copy-on-write snapshots

•Saves volume changes in files on the volume

•Uses defrag API to determine where the file is and where paging file is to avoid tracking their changes

21

Volume Shadow Copy Driver

(volsnap.sys)

Volume Shadow Copy Driver

(volsnap.sys)Mirror providerMirror provider

OracleOracle

SQLSQLVolume Shadow

Copy Service

Volume ShadowCopy Service

Backup Application

Backup Application

1. Backup application requests shadow copy

2. Writers told to freeze activity

3. Providers asked to create volume shadow copies

4. Writers told to resume (“thaw”) activity

Writers

Providers

5. Backup applicationsaves data from volume Shadow copies

Volume SnapshotsVolume Snapshots

22

Volsnap.sysVolsnap.sys

File System Driver

Volsnap.sys

a b c

a d …b c

Backup read of sector c

Snapshot

Backup Application Application

ShadowVolume

C:C:

Application read of sector cAll reads of sector d

23

Shadow Copies of Shared FoldersShadow Copies of Shared Folders

When enabled, Server When enabled, Server 2003 uses shadow copy 2003 uses shadow copy to periodically create to periodically create snapshots of volumessnapshots of volumes

Schedule and space used Schedule and space used is configurableis configurable

24

Shadow Copies on Shared FoldersShadow Copies on Shared Folders

Shadow copies are only Shadow copies are only exposed as network sharesexposed as network shares

Clients may install an Clients may install an Explorer extension that Explorer extension that integrates with the file server integrates with the file server and let’s themand let’s them

View the state of folders and View the state of folders and files within a snapshotfiles within a snapshot

Rollback individual folders and Rollback individual folders and files to a snapshotfiles to a snapshot

25

The PnP ManagerThe PnP Manager

In NT 4.0 each device driver is responsible for In NT 4.0 each device driver is responsible for enumerating all supported busses in search of enumerating all supported busses in search of devices they supportdevices they support

As of Windows 2000, the PnP Manager has bus As of Windows 2000, the PnP Manager has bus drivers enumerate their busses and inform it of drivers enumerate their busses and inform it of present devicespresent devices

If the device driver for a device not already If the device driver for a device not already present on the system, the PnP Manager in the present on the system, the PnP Manager in the kernel informs the user-mode PnP Manager to kernel informs the user-mode PnP Manager to start the Hardware Wizardstart the Hardware Wizard

26

The PnP ManagerThe PnP Manager

Once a device driver is located, Once a device driver is located, the PnP Manager determines ifthe PnP Manager determines if the driver is signed the driver is signed

If the driver If the driver is not signed, the system’s driver is not signed, the system’s driver signing policy determines whether signing policy determines whether or not the driver is installed or not the driver is installed

After loading a driver, the PnP Manager calls the driver’s AddDevice entry pointAfter loading a driver, the PnP Manager calls the driver’s AddDevice entry point

The driver informs the PnP Manager of the device’s resource requirementsThe driver informs the PnP Manager of the device’s resource requirements

The PnP Manager reconfigures other devices to accommodate the new deviceThe PnP Manager reconfigures other devices to accommodate the new device

27

The PnP ManagerThe PnP Manager

Enumeration is recursive, and directed by bus driversEnumeration is recursive, and directed by bus drivers

Bus drivers identify device on a busBus drivers identify device on a bus

As busses and devices are registered, a device tree is constructed, As busses and devices are registered, a device tree is constructed, and filled in with devicesand filled in with devices

Root

ACPI

PCI

USBVideo Disk

Key-board

Battery

Device Tree

28

Resource ArbitrationResource Arbitration

Devices require system hardware resources to function (e.g. IRQs, Devices require system hardware resources to function (e.g. IRQs, I/O ports)I/O ports)

The PnP Manager keeps track of hardware resource assignmentsThe PnP Manager keeps track of hardware resource assignments

If a device requires a resource that’s already been assigned, the If a device requires a resource that’s already been assigned, the PnP Manager tries to reassign resources in order to accommodate PnP Manager tries to reassign resources in order to accommodate

Example: Example:

1.1. Device 1 can use IRQ 5 or IRQ 6Device 1 can use IRQ 5 or IRQ 6

2.2. PnP Manager assigns it IRQ 5PnP Manager assigns it IRQ 5

3.3. Device 2 can only use IRQ 5Device 2 can only use IRQ 5

4.4. PnP Manager reassigns Device 1 IRQ 6PnP Manager reassigns Device 1 IRQ 6

5.5. PnP Manager assigns Device 2 IRQ 5PnP Manager assigns Device 2 IRQ 5

29

Plug and Play (PnP) State TransitionsPlug and Play (PnP) State Transitions

PnP manager recognizes hardware, allocates resources, loads PnP manager recognizes hardware, allocates resources, loads driver, notifies about config. changesdriver, notifies about config. changes

Not started

Started

Pending stop

Stopped

Pending remove

Surpriseremove

RemovedStart-devicecommand

Start-devicecommand

Query-stopcommand

Stopcommand

Query-removecommand

Surprise-removecommand

Removecommand

Removecommand

Device Plug and Play state transitionsDevice Plug and Play state transitions

30

The Power ManagerThe Power Manager

A system must have an ACPI-compliant BIOS for full compatibility (APM A system must have an ACPI-compliant BIOS for full compatibility (APM gives limited power support)gives limited power support)

A number of factors guide the Power Manager’s decision to change power A number of factors guide the Power Manager’s decision to change power state:state:

System activity level System activity level

System battery level System battery level

Shutdown, hibernate, or sleep requests from Shutdown, hibernate, or sleep requests from applications applications

User actions, such as pressing the power button User actions, such as pressing the power button

Control Panel power settingsControl Panel power settings

The system can go into low power modes, The system can go into low power modes, but it requires the cooperation of every but it requires the cooperation of every device driver - applications can provide their input as welldevice driver - applications can provide their input as well

31

The Power ManagerThe Power ManagerThere are different system power states:There are different system power states:

OnOn

Everything is fully onEverything is fully on

StandbyStandby

Intermediate statesIntermediate states

Lower standby states must consume less power than higher onesLower standby states must consume less power than higher ones

HibernatingHibernating

Save memory to disk in a file called hiberfil.sys in the root directory of the system Save memory to disk in a file called hiberfil.sys in the root directory of the system volumevolume

OffOff

All devices are offAll devices are off

Device drivers manage their own power levelDevice drivers manage their own power level

Only a driver knows the capabilities of their deviceOnly a driver knows the capabilities of their device

Some devices only have “on” and “off”, others have intermediate statesSome devices only have “on” and “off”, others have intermediate states

Drivers can control their own power independently of system powerDrivers can control their own power independently of system power

Display can dim, disk spin down, etc.Display can dim, disk spin down, etc.

32

Power ManagerPower Manager

based on the Advanced Configuration and Power Interface (ACPI)based on the Advanced Configuration and Power Interface (ACPI)

StateState Power ConsumptionPower Consumption Software ResumptionSoftware Resumption HW LatencyHW Latency

S0 (fully on)S0 (fully on) MaximumMaximum Not applicableNot applicable NoneNone

S1 (sleeping)S1 (sleeping) Less than S0,Less than S0,more than S2more than S2

System resumes where it left System resumes where it left off (returns to S0)off (returns to S0)

Less than 2 Less than 2 sec.sec.

S2 (sleeping)S2 (sleeping) Less than S1,Less than S1,more than S3more than S3

System resumes where it left System resumes where it left off (returns to S0)off (returns to S0)

2 or more 2 or more sec.sec.

S3 (sleeping)S3 (sleeping) Less than S2,Less than S2,processor is offprocessor is off

System resumes where it left System resumes where it left off (returns to S0)off (returns to S0)

Same as S2Same as S2

S4 (sleeping)S4 (sleeping) Trickle current to power Trickle current to power button and wake button and wake circuitrycircuitry

System restarts from System restarts from hibernate file and resumes hibernate file and resumes where it left off (returns to S0)where it left off (returns to S0)

Long and Long and undefinedundefined

S5 (fully off)S5 (fully off) Trickle current to Trickle current to power buttonpower button

System bootSystem boot Long and Long and undefinedundefined

System Power-State DefinitionsSystem Power-State Definitions

33

Troubleshooting I/O ActivityTroubleshooting I/O Activity

Filemon can be a great help to understand and troubleshooting I/O Filemon can be a great help to understand and troubleshooting I/O problemsproblems

Two basic techniques:Two basic techniques:

Go to end of log and look backwards to where problem occurred or is Go to end of log and look backwards to where problem occurred or is evident and focused on the last things doneevident and focused on the last things done

Compare a good log with a bad logCompare a good log with a bad log

Often comparing the I/O activity of a failing process with one that works Often comparing the I/O activity of a failing process with one that works may point to the problemmay point to the problem

Have to first massage log file to remove data that differs run to runHave to first massage log file to remove data that differs run to run

Delete first 3 columns (they are always different: line #, time, process Delete first 3 columns (they are always different: line #, time, process id)id)

Easy to do with Excel by deleting columnsEasy to do with Excel by deleting columns

Then compare with FC (built in tool) or Windiff (Resource Kit)Then compare with FC (built in tool) or Windiff (Resource Kit)

34

FilemonFilemon# - operation number# - operation number

Process: image name + process idProcess: image name + process id

Request: internal I/O request codeRequest: internal I/O request code

Result: return code from I/O operationResult: return code from I/O operation

Other: flags passed on I/O requestOther: flags passed on I/O request

35

Using FilemonUsing Filemon

Start/stop logging (Control/E)Start/stop logging (Control/E)

Clear display (Control/X)Clear display (Control/X)

Open Explorer window to folder containing file:Open Explorer window to folder containing file:

Double click on a line does thisDouble click on a line does this

Find – finds text within windowFind – finds text within window

Save to log fileSave to log file

Advanced modeAdvanced mode

Network optionNetwork option

36

What Filemon MonitorsWhat Filemon MonitorsBy default Filemon traces all file I/O to:By default Filemon traces all file I/O to:

Local non-removable mediaLocal non-removable media

Network sharesNetwork shares

Stores all output in listviewStores all output in listviewCan exhaust virtual memory in long Can exhaust virtual memory in long runsruns

You can limit captured data with history You can limit captured data with history depthdepth

You can limit what is monitored:You can limit what is monitored:What volumes to watch in Volumes menuWhat volumes to watch in Volumes menu

What paths and processes to watch in Filter dialogWhat paths and processes to watch in Filter dialog

What operations to watch in Filter dialog (reads, writes, successes What operations to watch in Filter dialog (reads, writes, successes and errors)and errors)

37

Filemon Filtering and HighlightingFilemon Filtering and HighlightingInclude and exclude filters are substring matches against Include and exclude filters are substring matches against the process and path columnsthe process and path columns

Exclude overrides include filterExclude overrides include filter

Be careful that you don’t exclude potentially useful dataBe careful that you don’t exclude potentially useful data

Capture everything and save the logCapture everything and save the log

Then apply filters (you can always reload the log)Then apply filters (you can always reload the log)

Highlight matches all columnsHighlight matches all columns

38

Basic vs Advanced ModeBasic vs Advanced Mode

Basic mode massages output to be sysadmin-Basic mode massages output to be sysadmin-friendly and target common troubleshootingfriendly and target common troubleshooting

Things you don’t see in Basic mode:Things you don’t see in Basic mode:

Raw I/O request namesRaw I/O request names

Various internal file system operationsVarious internal file system operations

Activity in the System processActivity in the System process

Page file I/OPage file I/O

Filemon file system activityFilemon file system activity

39

Understanding Disk ActivityUnderstanding Disk Activity

Use Filemon to see why you’re hard disk is crunchingUse Filemon to see why you’re hard disk is crunching

Process performance counters show I/O activity, but not to whereProcess performance counters show I/O activity, but not to where

System performance counters show which disks are being hit, but System performance counters show which disks are being hit, but not which files or which processnot which files or which process

Filemon pinpoints which file(s) are being accessed, by whom, Filemon pinpoints which file(s) are being accessed, by whom, and how frequentlyand how frequently

You can also use Filemon on a server to determine which You can also use Filemon on a server to determine which file(s) were being accessed most frequentlyfile(s) were being accessed most frequently

Import into Excel and make a pie chart by file name or operation Import into Excel and make a pie chart by file name or operation typetype

Move heavy-access files to a different disk on a different Move heavy-access files to a different disk on a different controllercontroller

40

Polling and File Change NotificationPolling and File Change NotificationMany applications respond to file and directory changesMany applications respond to file and directory changes

A poorly written application will “poll” for changesA poorly written application will “poll” for changes

A well-written application will request notification by the system of A well-written application will request notification by the system of changeschanges

Polling for changes causes performance degradationPolling for changes causes performance degradationContext switches including TLB flushContext switches including TLB flush

Cache invalidationCache invalidation

Physical memory usagePhysical memory usage

CPU usageCPU usage

Alternative: file change notification Alternative: file change notification

When you run Filemon on an idle system you should only see When you run Filemon on an idle system you should only see bursty system background activitybursty system background activity

Polling is visible as periodic accesses to the same files and Polling is visible as periodic accesses to the same files and directoriesdirectories

File change notification is visible as directory queries that have no File change notification is visible as directory queries that have no resultresult

41

Example: Word CrashExample: Word Crash

While typing in the document Word XP While typing in the document Word XP would intermittently close without any error would intermittently close without any error messagemessage

To troubleshoot ran Filemon on user’s To troubleshoot ran Filemon on user’s systemsystem

Set the history depth to 10,000Set the history depth to 10,000

Asked user to send Filemon log when Word Asked user to send Filemon log when Word exitedexited

42

Solution: Word CrashSolution: Word Crash

Working backwards, the first “strange” or Working backwards, the first “strange” or unexplainable behavior are the constant reads unexplainable behavior are the constant reads past end of file to MSSP3ES.LEXpast end of file to MSSP3ES.LEX

User looked up what .LEX file wasUser looked up what .LEX file was

Related to Word proofing toolsRelated to Word proofing tools

Uninstalled and reinstalled proofing tools & problem Uninstalled and reinstalled proofing tools & problem went awaywent away

43

Example: Useless Excel Error Example: Useless Excel Error MessageMessage

Excel reports an error “Unable to read Excel reports an error “Unable to read file" when startingfile" when starting

44

Solution: Useless Excel Error MessageSolution: Useless Excel Error MessageFilemon trace shows Excel reading file in Filemon trace shows Excel reading file in XLStart folder XLStart folder

All Office apps autoload files in their start foldersAll Office apps autoload files in their start folders

Should have reported:Should have reported:

Name and location of fileName and location of file

Reason why it didn’t like itReason why it didn’t like it

45

Further ReadingFurther Reading

Mark E. Russinovich and David A. Solomon, Mark E. Russinovich and David A. Solomon, Microsoft Windows Internals, 4th Edition, Microsoft Windows Internals, 4th Edition, Microsoft Press, 2004.Microsoft Press, 2004.

I/O Processing (from pp. 561)I/O Processing (from pp. 561)

The Plug and Play (PnP) Manager (from pp. 590)The Plug and Play (PnP) Manager (from pp. 590)

The Power Manager (from pp. 607)The Power Manager (from pp. 607)

Troubleshooting File System Problems (from pp. Troubleshooting File System Problems (from pp. 711)711)

46

Source Code ReferencesSource Code References

Windows Research Kernel sourcesWindows Research Kernel sources

\base\ntos\io – I/O Manager\base\ntos\io – I/O Manager

\base\ntos\inc\io.h – additional structure/type \base\ntos\inc\io.h – additional structure/type definitionsdefinitions

\base\ntos\verifer – Driver Verifier\base\ntos\verifer – Driver Verifier

\base\ntos\inc\verifier.h – additional structure/type \base\ntos\inc\verifier.h – additional structure/type definitionsdefinitions