solaris 10 kernel solaris 10 kernel presentation

Download Solaris 10 Kernel Solaris 10 Kernel Presentation

Post on 13-Apr-2015

70 views

Category:

Documents

13 download

Embed Size (px)

DESCRIPTION

Solaris 10 Kernel Solaris 10 Kernel Presentation

TRANSCRIPT

SOLARIS

TM

Kernel

Performance, Observability & DebuggingRichard McDougallDistinguished Engineer Performance and Availability Engineering Sun Microsystems, Inc. r@sun.com

James MauroSenior Staff Engineer Performance and Availability Engineering Sun Microsystems, Inc. james.mauro@sun.com

This tutorial is copyright 2005 by Richard McDougall and James Mauro. It may not be used in whole or part for commercial purposes without the express written consent of Richard McDougall and James Mauro

About The Instructors Richard McDougall is a Distinguished Engineer in the Performance and Availability Engineering group at Sun Microsystems, where he focuses on large systems architecture, performance, measurement and observability. Richard's interests and expertise include the development of tools for measuring and Sizing Solaris systems. Among his numerous contributions, Richard designed and implemented numerous enhancements to the Solaris kernel virtual memory subsystem, and file system IO layer. Jim Mauro is a Senior Staff Engineer in the Performance and Availability Engineering group at Sun Microsystems, where he focuses on availability benchmarking and system performance tuning. Jim's past efforts include developing a framework for measuring system availability, individual availability benchmarks, improving SunCluster availability and establishing company-wide metrics for assessing system availability.

Richard and Jim authored Solaris Internals: Core Kernel Architecture, Prentice Hall, 2001. ISBN 0-13-022496-0 r@sun.com james.mauro@sun.comCopyright 2005 Richard McDougall & James Mauro

USENIX '05 Anaheim, Ca.

Credits Phil Harman: Multi-threading diagrams and Solaris introduction Bryan Cantril, Mike Shapiro, Adam Leventhal: Solaris dtrace tutorial Scott Fehrman: Virtual memory graphics Kevin Sheehan: IO Topology slides

Copyright 2005 Richard McDougall & James Mauro

USENIX '05 Anaheim, Ca.

Agenda Day 1

Session 1 - 9:00AM to 10:30PM

Goals, non goals and assumptions Solaris Kernel Overview & Features Observability & Tracing Tools & Utilities Memory

Session 2 - 11:00PM to 12:30PM

Virtual Memory Physical Memory Memory dynamics Performance and Observability Memory Resource Management

Copyright 2005 Richard McDougall & James Mauro

USENIX '05 Anaheim, Ca.

Agenda Day 1 (cont)

Session 3 - 2:00PM to 3:30PM

Processes, threads & scheduling

The Solaris Multhreaded Process Model The Dispatcher & Scheduling Classes Performance & Observability Processor Controls and Binding

Session 4 - 4:00PM to 5:30PM

File Systems and I/O I/O Overview

The Solaris VFS/Vnode Model UFS The Solaris Unix File System Performance & Observability

Copyright 2005 Richard McDougall & James Mauro

USENIX '05 Anaheim, Ca.

Agenda Day 2

Session 1 - 9:00AM to 10:30PM

DTrace A Deeper Dive A System View Traps & Interrupts Advanced Memory Topics

Session 2 - 11:00PM to 12:30PM

Memory monitoring and measuring Utilizing and tuning large memory

Copyright 2005 Richard McDougall & James Mauro

USENIX '05 Anaheim, Ca.

Agenda Day 2 (cont)

Session 3 - 2:00PM to 3:30PM

Processes, threads & scheduling

A Deeping Dive The Runtime Linker Watching Processes with Dtrace Process/Thread Lab

Session 4 - 4:00PM to 5:30PM Disk I/O Performance File System Performance Network Attached Storage File System Performance Characterization ZFS Resource Management Large System PerformanceUSENIX '05 Anaheim, Ca.

Copyright 2005 Richard McDougall & James Mauro

Goals, Non-goals & Assumptions

Goals

Non-goals

Architectural overview of the Solaris kernel Drill down into key subsystems The tools what they are, what they do, when and how to use them Correlate performance & observability to key functions Resource control & management framework Detailed look at core kernel algorithms Networking internals General familiarity with the Solaris environment General familiarity with operating systems conceptsUSENIX '05 Anaheim, Ca.

Assumptions

Copyright 2005 Richard McDougall & James Mauro

Why Performance, Observability & Debugging?

Reality, what a concept

Chasing performance problems

Sometimes they are even well defined My app should be doing X, but it's doing Y

Chasing pathological behaviour

It's only doing it sometimes

Understand utilization

Resource consumption

CPU, Memory, IO

Capacity planning

90% of system activity falls into one of the above categories, for a variety of roles

In general, attaining a good understanding of the system, the workload, and how they interact

Copyright 2005 Richard McDougall & James Mauro

Admins, DBA's, Developers, etc...

USENIX '05 Anaheim, Ca.

Before You Begin...Would you tell me, please, which way I ought to go from here? asked Alice That depends a good deal on where you want to get to said the Cat I don't much care where... said Alice Then it doesn't matter which way you go said the Cat

Lewis Carroll Alice's Adventures in Wonderland

Copyright 2005 Richard McDougall & James Mauro

USENIX '05 Anaheim, Ca.

General Methods & Approaches

Define the problem

System View

In terms of a business metric Something measurable Resource usageCPU, Memory, Network, IO

Process View

Execution profileWhere's the time being spent

Drill down depends on observations & goals

May lead to a thread view

The path to root-cause has many forks bottlenecks move

Moving to the next knee-in-the-curveUSENIX '05 Anaheim, Ca.

Copyright 2005 Richard McDougall & James Mauro

Solaris Kernel Features

Dynamic Multithreaded Preemptive Multithreaded Process Model Multiple Scheduling Classes

Tightly Integrated File System & Virtual Memory Virtual File System 64-bit kernel

Including realtime support

Resource Management Service Management & Fault Handling Integrated NetworkingUSENIX '05 Anaheim, Ca.

32-bit and 64-bit application support

Copyright 2005 Richard McDougall & James Mauro

The 64-bit RevolutionSolaris 2.6 ILP32 Apps ILP32 Libs ILP32 Kernel ILP32 Drivers 32-bit HWCopyright 2005 Richard McDougall & James Mauro

Solaris 7, 8, 9, 10, ... ILP32 Apps ILP32 Libs ILP32 Kernel* ILP32 Drivers* 32-bit HW* LP64 Apps LP64 Libs LP64 Kernel LP64 Drivers 64-bit HWUSENIX '05 Anaheim, Ca.

* Solaris 10: 64-bit kernel only on SPARC

Solaris 8A Few Selected Highlights

A new 1:1 threads implementation

Page cache enhancements (segmap)

/usr/lib/lwp/libthread.so Cyclic page cache

/dev/poll for scalable I/O Modular debugging with mdb(1) You want statistics?

UFS Direct I/O

kstat(1M), prstat(1M), lockstat(1M), busstat(1M), cpustat(1M), ...

Copyright 2005 Richard McDougall & James Mauro

USENIX '05 Anaheim, Ca.

The Multithreading RevolutionSolaris 2.0 Solaris 8 Solaris 8, 9, 10, ...

dispatcher

dispatcher

processors

processors

processCopyright 2005 Richard McDougall & James Mauro

User thread

LWP

Kernel threadUSENIX '05 Anaheim, Ca.

Solaris 9A Subset of the 300+ New Features Manageability Availability Security

Scalability

Solaris Containers SolarisTM 9 Resource Manager IPQoS SolarisTM Volume Manager (SVM) Soft Disk Partitions Filesystem for DBMS UFS Snapshots SolarisTM Flash SolarisTM Live Upgrade 2.0 Patch Manager Product Registry Sun ONE DS integration Legacy directory proxy Secure LDAP client Solaris WBEM Services Solaris instrumentation FRU ID SunTM Management Center

Solaris Live Upgrade 2.0 Dynamic Reconfiguration Sun StorEdgeTM Traffic Manager Software IP Multipathing Reconfiguration Coordination Manager Driver Fault Injection Framework Mobile IP Reliable NFS TCP timers

Copyright 2005 Richard McDougall & James Mauro

IPv6 IPSec v4 and v6 Thread enhancements SunScreen Firewall Memory optimization Enhanced RBAC Advanced page coloring Kerberos V5 Mem Placement Optimization IKE Multi Page Size Support PAM enhancements Hotspot JVM tuning Secure sockets layer (SSL) NFS performance increase SolarisTM Secure Shell UFS Direct I/O Extensible password Dynamic System Domains encryption Enhanced DNLC SolarisTM RSM API Security Toolkit J2SETM 1.4 software with TCP Wrappers 64-bit and IPv6 Kernel and user-level NCA enhancements encryption frameworks . . . and more: Random number generator Compatibility Guarantee SmartCard APIs Java Support Linux Compatibility Network Services G11N and Accessibility GNOME Desktop Anaheim, Ca. USENIX '05

Solaris 10The Headline Grabbers

Solaris Containers (Zones) Solaris Dynamic Tracing (dtrace) Predictive Self Healing

Process Rights Management Premier x86 support Optimized 64-bit Opteron support (x64) Zetabyte Filesystem (ZFS) ... and much, much more!

System Management Framework Fault Management Architecture

Copyright 2005 Richard McDougall & James Mauro

USENIX '05 Anaheim, Ca.

Solaris Kernel OverviewSystem Call InterfaceProcesses & Threads Schedulertimeshare interactive realtime fair share fixed priority

Kernel Services Clocks & Timers etc

Virtual File System Framework

Networking framework & services

UFS NFS

ProcFS

. .Sp