SDA Extensions

Download SDA Extensions

Post on 13-Oct-2015

32 views

Category:

Documents

1 download

Embed Size (px)

DESCRIPTION

openvms sda

TRANSCRIPT

<ul><li><p>Using SDA Extensions to Troubleshoot System Hangs and Performance Issues</p><p>Presented by </p><p>Jim Mehlhopmehlhop@parsec.com</p><p>www.parsec.com | 888-4-PARSEC</p><p>To Download</p><p>http://www.parsec.com/public/SDA_Extensions.pdf</p></li><li><p> Activating SDA</p><p> Symbols</p><p> Read Global Symbols from System Loadable Images</p><p> Read Global Symbols from Any Object Module</p><p>Topics</p><p> Format Instructions and Blocks of Data</p></li><li><p> Copy the System Dump File</p><p> Send Output to a File or Device</p><p> SDA Extensions What are they</p><p> How they work</p><p> They are undocumented, getting help can be a challenge</p><p>Topics (contd)</p></li><li><p>Topics (contd)</p><p> Clue Example</p><p> Example of using SDA PCS to troubleshoot a hung system</p><p> Examples using SDA SPL to troubleshoot high MP SYNCHRONIZATION time</p><p> Example using FLT to isolate source of alignment faults, a major source of performance issues on Integrity Servers.</p><p> Additional extensions</p></li><li><p>Activating SDA</p><p> There are two ways of getting into SDA. One is to examine the live system and the other is to analyze a crash dump file. In both cases it reads a basic symbol table: REQSYSDEF.STB</p><p> To analyze a live system, issue the following: $ analyze/system</p><p> Need the CMKRNL privilege Since it is a live system, things change REMEMBER that</p><p> To analyze a crash dump, issue the following: $ analyze/crash dump-filespec</p><p> Need read access to the dump file Everything is static</p></li><li><p>Activating SDA (contd)</p><p> Either way you start SDA, it always looks for an initialization file.</p><p> First it looks for a logical SDA$INIT</p><p> Second it looks in the current users default login directory for a file named sda.init</p><p> The purpose for this file is to execute SDA commands that you ALWAYS want to execute. </p><p> An example of this would be commands to read in symbols. We will be talking about symbols shortly.</p></li><li><p>Reading Symbols</p><p> Symbols are read both automatically and under user control. Automatically </p><p> When SDA gets activated, it reads reqsysdef.stb from the appropriate directory for the current architecture.</p><p> User control SDA&gt; read/exec </p><p> Reads symbols from system loadable image list SDA&gt; read </p><p> Default file type is .stb, however it reads .obj files as well VMS supplies .stb files for most loadable_images and subsystems</p><p> $DIR SYS$LOADABLE_IMAGES:*.stb (Alpha) Subsystem STBs DCLDEF, F11BXQPDEF, IMGDEF, </p><p>IODEF, RMSDEF, SCSDEF, and a system-wide SYSDEF</p></li><li><p>Reading Symbols (contd)</p><p> Users can supplement the VMS symbols</p><p> Create a Macro source file with needed definitions</p><p> $ edit symbols_example.mar</p><p> Compile that source</p><p> $ mac symbols_example.mar</p></li><li><p>Reading Symbols Examples #3Reading User-Created Object File</p><p>There is an offset in a drivers Function Decision Table that indicates which functions support 64 bit addressing </p><p>named FDT$Q_OK64BIT by default VMS does not make that symbol available to you. So we create a macro </p><p>source program that pulls the FDT symbols in from the Macro library.</p><p>CLASS3$ type symbols_example.mar</p><p>.LIBRARY /SYS$LIBRARY:LIB.MLB/</p><p>.LIBRARY /SYS$LIBRARY:STARLET.MLB/</p><p>$FDTDEF GLOBAL</p><p>$PFNDEF GLOBAL ;PFN DATA RECORD(Hopefully)</p><p>$PFREEDEF GLOBAL ;FREE POOL PACKET DEFINITIONS</p><p>$PRDEF GLOBAL ;PROCESSOR REGISTER DEFINITIONS</p><p>$PSIGDEF GLOBAL ;CALL SIGNATURE DEFINITIONS</p><p>$PTEDEF GLOBAL ; Page Table Entry DEFINITIONS</p><p>$PTRDEF GLOBAL ; V7.3 PAGEFILE POINTER STRUCURE</p><p>.END</p><p>CLASS3$ mac symbols_example.mar</p><p>SDA&gt; sho sym FDT$Q_OK64BIT</p><p>%SDA-E-BADSYM, unknown symbol "FDT$Q_OK64BIT"</p><p>SDA&gt; read SYMBOLS_EXAMPLE.obj</p><p>%SDA-I-READSYM, 1394 symbols read from STAFF_DISK:[MEHLHOP]SYMBOLS_EXAMPLE.OBJ;1</p><p>SDA&gt; sho sym FDT$Q_OK64BIT</p><p>FDT$Q_OK64BIT = 00000000.00000108</p></li><li><p>SDA Commands Format</p><p> The format command allows you to display a block of memory that is defined as a data structure. </p><p> It relies on the convention that the byte at offset 10(decimal) 0A(hex) defines which data structure the block represents.</p><p> If there are symbols that represent that data structure, then SDA will display those names and the data in a formatted output as displayed in the next slide.</p><p> If the block does not follow the convention or does not have a value that SDA understands, then SDA will either not format it or format it as the wrong type of structure.</p></li><li><p>SDA Commands Format (contd)</p><p>SDA&gt; sho proc ! To define PCB symbol</p><p>.</p><p>.</p><p>.</p><p>SDA&gt; ex pcb+0A</p><p>%SDA-W-UNALIGNED, unaligned address FFFFFFFF.81E8200A; converting to aligned address</p><p>PCB+00008: 000C0580 "...."</p><p>SDA&gt; !0c=PCB</p><p>SDA&gt; format pcb</p><p>%SDA-E-NOSYMBOLS, no "PCB" symbols found to format this block</p><p>SDA&gt; read sysdef</p><p>SDA&gt; format pcb</p><p>FFFFFFFF.81E82000 PCB$L_SQFL 818099E8 SCH$AQ_COMT+001A4</p><p>FFFFFFFF.81E82004 PCB$L_SQBL 818099E8 SCH$AQ_COMT+001A4</p><p>FFFFFFFF.81E82008 PCB$W_SIZE 0580 </p><p>FFFFFFFF.81E8200A PCB$B_TYPE 0C</p><p>FFFFFFFF.81E8200B </p><p>00 </p><p>.</p><p>.</p><p>.</p></li><li><p>SDA Commands Format (contd)</p><p>SDA&gt; format phd !This structure does NOT follow convention note type=0</p><p>%SDA-E-INVBLKTYP, invalid block type in specified block</p><p>SDA&gt; form phd/type=phd</p><p>FFFFFFFF.82866000 PHD$Q_PRIVMSK 00000040.D05DC205 </p><p>FFFFFFFF.82866008 PHD$W_SIZE 0000 </p><p>FFFFFFFF.8286600A PHD$B_TYPE 00</p><p>FFFFFFFF.8286600B 00 </p><p>FFFFFFFF.8286600C PHD$L_WSLIST 00000001 </p><p>FFFFFFFF.82866010 PHD$L_WSLOCK 0000000A </p><p>FFFFFFFF.82866014 PHD$L_WSDYN 00000012 </p><p>FFFFFFFF.82866018 PHD$L_WSNEXT 00000263 </p><p>FFFFFFFF.8286601C PHD$L_WSLAST 000002E2 </p><p>FFFFFFFF.82866020 PHD$L_WSEXTENT 00000400 </p><p>FFFFFFFF.82866024 PHD$L_WSQUOTA 00000120 </p><p>.</p><p>.</p><p>.</p></li><li><p>SDA Commands Misc.</p><p> SDA&gt; show executive</p><p> SDA&gt; map 810B8050</p><p> SDA&gt; show cluster</p><p> SDA&gt; show lan</p><p> SDA&gt; copy dev::[dir]filename</p><p> SDA&gt; set output dev::[dir]filename</p><p> SDA&gt; set log dev::[dir]filename</p></li><li><p>Extensions What are they</p><p> Special purpose code that calls SDA callable routines to examine memory under the protection of SDA.</p><p> They are searched for whenever you issue an SDA command that is not a native SDA command.</p><p> Searches for a logical name in the format of whatever the command was plus $SDA. For example SDA&gt; PCS show trace will look for the logical PCS$SDA. If that logical does not exist it will look in the SYS$LIBRARY directory for the image PCS$SDA.</p></li><li><p>Extensions What are they (cont)</p><p> Anyone can create an SDA extension. The first extension was CLUE. The second one was written by an ISV to support troubleshooting their application.</p><p> The extensions supplied by OpenVMS can be identified by issuing the following command $DIRECTORY SYS$LIBRARY:*SDA.EXE</p></li><li><p>Extensions What are they (cont)</p><p> Extensions are version/patch/Architecture specific</p><p> Alpha</p><p>SYS6$ show system/noprocess</p><p>OpenVMS V8.3 on node SYS6 12-SEP-2011 16:10:23.94 Uptime 0 00:31:38</p><p>SYS6$ dir sys$share:*$sda.exe</p><p>Directory SYS$COMMON:[SYSLIB]</p><p>CLUE$SDA.EXE;1 CNX$SDA.EXE;1 DECDTM$SDA.EXE;1 DKLOG$SDA.EXE;1</p><p>EXC$SDA.EXE;1 FC$SDA.EXE;1 FLT$SDA.EXE;1 IO$SDA.EXE;1</p><p>IPC$SDA.EXE;1 ISCSI$SDA.EXE;1 LAN$SDA.EXE;1 LCK$SDA.EXE;1</p><p>LNM$SDA.EXE;1 MTX$SDA.EXE;1 OCLA$SDA.EXE;1 PCS$SDA.EXE;1</p><p>PE$SDA.EXE;1 PKM$SDA.EXE;1 PRF$SDA.EXE;1 PSH$SDA.EXE;1</p><p>PTHREAD$SDA.EXE;1 PWIP$SDA.EXE;1 RMS$SDA.EXE;1 SHAD$SDA.EXE;1</p><p>SPL$SDA.EXE;1 TCPIP$SDA.EXE;1 TQE$SDA.EXE;1 TR$SDA.EXE;1</p><p>USB$SDA.EXE;1 XFC$SDA.EXE;1 XX$SDA.EXE;1</p><p>Total of 31 files.</p><p>SYS6$</p></li><li><p> Integrity</p><p>SYS226$ show system/noprocess</p><p>OpenVMS V8.3-1H1 on node SYS226 12-SEP-2011 16:15:05.95 Uptime 49 04:31:46</p><p>SYS226$ dir sys$share:*$sda.exe</p><p>Directory SYS$COMMON:[SYSLIB]</p><p>CLUE$SDA.EXE;1 CNX$SDA.EXE;1 DECDTM$SDA.EXE;1 DKLOG$SDA.EXE;1</p><p>EXC$SDA.EXE;1 FC$SDA.EXE;1 FLT$SDA.EXE;1 IO$SDA.EXE;1</p><p>IPC$SDA.EXE;1 ISCSI$SDA.EXE;1 LAN$SDA.EXE;1 LCK$SDA.EXE;1</p><p>LNM$SDA.EXE;1 MTX$SDA.EXE;1 PCS$SDA.EXE;1 PE$SDA.EXE;1</p><p>PKM$SDA.EXE;1 PKR$SDA.EXE;1 PRF$SDA.EXE;1 PSH$SDA.EXE;1</p><p>PTHREAD$SDA.EXE;1 PWIP$SDA.EXE;1 RMS$SDA.EXE;1 SHAD$SDA.EXE;1</p><p>SPL$SDA.EXE;1 TCPIP$SDA.EXE;1 TQE$SDA.EXE;1 TR$SDA.EXE;1</p><p>USB$SDA.EXE;1 XFC$SDA.EXE;1 XX$SDA.EXE;1</p><p>Total of 31 files.</p><p>SYS226$</p><p>Extensions What are they (cont)</p></li><li><p> IntegrityCLASS3$ show system/noprocess</p><p>OpenVMS V8.4 on node CLASS3 12-SEP-2011 15:58:48.80 Uptime 5 23:26:10</p><p>CLASS3$ dir sys$share:*$sda.exe</p><p>Directory SYS$COMMON:[SYSLIB]</p><p>CLUE$SDA.EXE;1 CNX$SDA.EXE;1 DECDTM$SDA.EXE;1 DKLOG$SDA.EXE;1</p><p>EXC$SDA.EXE;1 FC$SDA.EXE;1 FLT$SDA.EXE;1 GSP$SDA.EXE;1</p><p>ICC$SDA.EXE;1 IO$SDA.EXE;1 IPC$SDA.EXE;1 ISCSI$SDA.EXE;1</p><p>LAN$SDA.EXE;1 LCK$SDA.EXE;1 LES$SDA.EXE;1 LNM$SDA.EXE;1</p><p>MTX$SDA.EXE;1 NET$SDA.EXE;1 PCS$SDA.EXE;1 PE$SDA.EXE;1</p><p>PKD$SDA.EXE;1 PKM$SDA.EXE;1 PKR$SDA.EXE;1 PRF$SDA.EXE;1</p><p>PSH$SDA.EXE;1 PTHREAD$SDA.EXE;1 PWIP$SDA.EXE;1 RMS$SDA.EXE;1</p><p>SHAD$SDA.EXE;1 SPL$SDA.EXE;1 SWIS$SDA.EXE;1 TCPIP$SDA.EXE;1</p><p>TQE$SDA.EXE;1 TR$SDA.EXE;1 USB$SDA.EXE;1 XFC$SDA.EXE;1</p><p>XX$SDA.EXE;1</p><p>Total of 37 files.</p><p>Extensions What are they (cont)</p></li><li><p>How they work</p><p> When you use an extension command SDA loads the extension into YOUR process address space.</p><p> They may then collect for display information that is contained in the OS</p><p> Many of them also include an image in the SYS$LOADABLE_IMAGES directory that gets loaded into system space (GH) with a load subcommand to the extension. This image will stay in memory until either unloaded or the system reboots</p></li><li><p>How they work(cont)</p><p>For those extensions that collect information that is not normally collected by the OS the extension will also allocate a ring buffer in nonpaged pool.</p><p>This allows for you to start collection, log off, and later examine the collection after an event (including a system crash).</p></li><li><p>How they work(cont)</p><p>Process address space System address space</p><p>SDA.EXE</p><p>Shareable images</p><p>xxx$SDA</p><p>xxx$DEBUG</p><p>(system loadable image)</p><p>Ring Buffer</p></li><li><p>How they work(cont)CLASS8$ an/sy</p><p>OpenVMS system analyzer</p><p>SDA&gt; pcs load</p><p>PCS$DEBUG load status = 00000001</p><p>SDA&gt; sho exec pcs$debug</p><p>VMS Executive layout</p><p>--------------------</p><p>Image Base End Length Image Offset SymVec</p><p>--------------------------------------- ----------------- ----------------- ----------------- ----------------- --------</p><p>PCS$DEBUG </p><p>Code FFFFFFFF.818E6000 FFFFFFFF.818E76EF 00000000.000016F0 00000000.00010000</p><p>Data (read only) FFFFFFFF.886FD200 FFFFFFFF.886FD367 00000000.00000168 00000000.00020000</p><p>Data (read/write) FFFFFFFF.886FD400 FFFFFFFF.886FD40F 00000000.00000010 00000000.00030000</p><p>Short data (read/write) FFFFFFFF.886FD600 FFFFFFFF.886FD84F 00000000.00000250 00000000.00060000</p><p>Linked 29-JUN-2006 23:20 LDRIMG = 88FA7A40 SeqNum = 000000B6 GP = FFFFFFFF.888FD600</p><p>SDA&gt; pcs unload</p><p>PCS$DEBUG unload status = 00000001</p><p>SDA&gt; sho exec pcs$debug</p><p>No loadable image matching "PCS$DEBUG" found</p><p>SDA&gt;</p></li><li><p>Getting help</p><p> Some have limited help that you get by specifying just the command and no qualifiers (subcommand)</p><p> Some have built in help </p><p> Some have a help library</p></li><li><p>Getting help (cont)</p><p>SDA&gt; flt</p><p>Alignment Fault Tracing Utility FLT commands:</p><p>FLT LOAD</p><p>FLT UNLOAD</p><p>FLT START TRACE [/BUFFER=pages] </p><p>[/BEGIN=pc_range_low] </p><p>[/END=pc_range_high]</p><p>[/MODE=(K,E,S,U)] (default is ALL modes)</p><p>[/INDEX=pid] (default is ALL processes)</p><p>FLT STOP TRACE </p><p>FLT SHOW TRACE [/SUMMARY]</p></li><li><p>SDA&gt; pe help</p><p>PE$SDA Extension on CLASS2 (AlphaServer 4100 5/400 4MB) crash at 24-APR-2007 15:15:58.06</p><p>----------------------------------------------------------------------------------------</p><p>PE</p><p>This command executes the PEDRIVER specific SDA extension image</p><p>which is a collection of PEDRIVER specific commands which mirror</p><p>the display functions of SYS$SYSTEM:SCACP.EXE in SDA.</p><p>By default this image is loaded from SYS$SHARE. However, it can</p><p>be redirected with the following logical:</p><p>$ define pe$sda [dir-spec]pe$sda.exe</p><p>Format:</p><p>SDA&gt; PE command</p><p>Additional information available:</p><p>Press RETURN for more.</p><p>SDA&gt; </p><p>PE$SDA Extension on CLASS2 (AlphaServer 4100 5/400 4MB) crash at 24-APR-2007 15:15:58.06</p><p>----------------------------------------------------------------------------------------</p><p>BUS CHANNEL LAN_DEVICE TRACE HELP VC</p><p>PE Subtopic? vc</p><p>Getting help(cont)</p></li><li><p>PE Subtopic? vc</p><p>PE</p><p>VC</p><p>Displays PEDRIVER virtual circuit data. Each VC is NISCACP</p><p>communications path between the local system and a remote</p><p>system comprised of a set of channels.</p><p>Format</p><p>PE VC</p><p>Additional information available:</p><p>Additional information available:</p><p>Press RETURN for more.</p><p>SDA&gt; </p><p>PE$SDA Extension on CLASS2 (AlphaServer 4100 5/400 4MB) crash at 24-APR-2007 15:15:58.06</p><p>----------------------------------------------------------------------------------------</p><p>Parameters Qualifiers</p><p>/ALL /COUNTERS /SUMMARY /VC /n</p><p>Example</p><p>PE VC Subtopic? exam</p><p>PE</p><p>VC</p><p>Example</p><p>SDA&gt; PE VC/ERRORS/VC=FOO</p><p>Displays VC error counters for all VCs whose name (remote node name)</p><p>starts with FOO.</p><p>PE VC Subtopic? </p><p>PE Subtopic? </p><p>Topic? </p><p>Getting help(cont)</p></li><li><p>SDA&gt; pwip</p><p>SDA:PWIP&gt; help</p><p>HELP</p><p>DEBUG [ON]</p><p>DEBUG OFF</p><p>DEBUG BREAKPOINT</p><p>PWIP HEADER</p><p>PWIP DATAGRAMS [ BRIEF | FULL | RAW | NUM ]</p><p>PWIP LISTENERS [ BRIEF | FULL | RAW | NUM ]</p><p>PWIP SESSIONS [ BRIEF | FULL | RAW | NUM ]</p><p>PWIP [ALL] ( BRIEF of Datagrams, Listeners &amp; Sessions)</p><p>PWIP TRACE</p><p>PWIP DEFAULT [ BRIEF | FULL | RAW ] (When NUM specified)</p><p>SDA:PWIP&gt; </p><p>Getting help(cont)</p></li><li><p>SDA&gt; spawn /nowait dir sys$help:*sda.hlb</p><p>SDA&gt; </p><p>Directory SYS$COMMON:[SYSHLP]</p><p>SDA.HLB;1 TCPIP$SDA.HLB;1 XFC$SDA.HLB;1 </p><p>Total of 3 files.</p><p>SDA&gt; tcpip help</p><p>TCPIP</p><p>Sorry, no documentation on TCPIP</p><p>Additional information available:</p><p>About_TCPIP$SDA Development_Qualifiers READ SEARCH</p><p>SHOW SYSCONFIG TAG FORMAT</p><p>Topic? devel</p><p>Sorry, no documentation on DEVEL</p><p>Additional information available:</p><p>TCPIP</p><p>Topic? tcpip</p><p>Getting help(cont)</p></li><li><p>Topic? tcpip</p><p>TCPIP</p><p>The TCPIP verb prefixes all of the TCPIP (TCP/IP Services for</p><p>The TCPIP verb prefixes all of the TCPIP (TCP/IP Services for</p><p>Press RETURN for more.</p><p>SDA&gt; </p><p>OpenVMS) SDA extension's commands.</p><p>Format:</p><p>TCPIP [qualifiers]</p><p>TCPIP FORMAT ...</p><p>TCPIP READ ...</p><p>TCPIP SEARCH ...</p><p>TCPIP SHOW ...</p><p>TCPIP SYSCONFIG ...</p><p>TCPIP TAG ...</p><p>Additional information about the TCPIP extension can be found</p><p>under the `About_TCPIP$SDA' topic.</p><p>Additional information available:</p><p>About_TCPIP$SDA Development_Qualifiers READ SEARCH</p><p>SHOW SYSCONFIG TAG FORMAT</p><p>Getting help(cont)</p></li><li><p>SDA&gt; tcpip help format </p><p>TCPIP</p><p>FORMAT</p><p>Formats the block of memory specified by the "address" parameter.</p><p>To do this the "MALLOC_HDR", located at negat...</p></li></ul>