week 1: startup, dispatcher, and enqueue services unit 1 ... · start and stop process of sap...
TRANSCRIPT
Week 1: Startup, Dispatcher, and Enqueue Services
Unit 1: Fundamentals of the SAP Start Service
2PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Fundamentals of the SAP Start ServiceUsage (1)
The SAP start service is a key element of SAP
systems as of SAP NetWeaver 7.0.
The process is called sapstartsrv.exe on
Windows, and sapstartsrv on UNIX platforms.
This process is a service in Windows operating
systems; in UNIX operating systems, it is
scheduled as a daemon.
There is a separate sapstartsrv process for
each instance, which is responsible for starting,
stopping, and monitoring the instance.
SAP Host Agent also has its own SAP start
service.
3PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Fundamentals of the SAP Start ServiceUsage (2)
The sapstartsrv program reads the instance
profile at the beginning. The profile defines
which steps must be performed during the start
process, and which processes of the instance
are started by sapstartsrv.
The profile defines that the database should be
started if it is not yet running.
Normally, the entries for the starting procedure
in instance profiles are not changed; if there
are any changes, sapstartsrv needs to be
restarted.
4PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Fundamentals of the SAP Start ServiceFunctions
The SAP start service provides the following functions
for monitoring SAP systems, instances, and
processes:
Starting and stopping
Monitoring the runtime state
Reading logs, traces, profiles, configuration files
Technical information, such as network ports, active sessions,
thread lists, etc.
These services are provided on the SAPControl SOAP
Web service, and used by SAP monitoring tools like the
SAP Microsoft Management Console (SAP MMC) or SAP
Management Console (SAP MC).
You can get the list of all options and so-called Web
methods by running the ‘sapcontrol –h’ or ‘sapcontrol’
command.
5PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Fundamentals of the SAP Start ServicePorts (1)
The start service binds to the following ports:
HTTP port 5<instancenumber>13 by default
(or sapctrl<instancenumber> in /etc/services)
HTTPS port 5<instancenumber>14 by default
(or sapctrls<instancenumber> in /etc/services)
Port information is included in sapstartsrv.log, in the work folder of every instance.
6PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Fundamentals of the SAP Start ServicePorts (2)
If critical functions (“protected methods”, such as
restarting an instance) are not called using a
trusted connection, they require authentication
with an operating system user and password.
On UNIX platforms, trusted connections go through
UNIX domain sockets (/tmp/.sapstream<port-no>)
On Windows, they go through “Windows named pipe”
(\\<host>\pipe\sapcontrol_<instancenumber>)
7PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Fundamentals of the SAP Start ServiceStartup process (1)
An SAP instance is started differently on
UNIX and Windows platforms:
UNIX
The startup procedure is initiated by the start
service, which ideally is started by the sapinit
script when the operating system is booted.
Afterwards, sapstartsrv triggers sapstart to
start the system.
The SAP start service definition is available
in the /usr/sap/sapservices file.
8PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Fundamentals of the SAP Start ServiceStartup process (2)
An SAP instance is started differently on
UNIX and Windows platforms:
Windows
On Windows, the start service is started
automatically when the operating system is
booted. The instance is started directly by
the start service.
The SAP start service definition is based on
Windows Registry entries.
9PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Fundamentals of the SAP Start ServiceStartup process (3)
The start service reads the instance profile and executes all commands in the instance profile that
contain an Execute_xx statement.
It then starts the processes of the SAP instance, respecting the sequential numbering of
Start_Program_xx statements.
Execute_xx and Start_Program_xx has its own, unique, sequential numbering, starting with 00.
10PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Fundamentals of the SAP Start ServiceStart and Stop process of SAP NetWeaver AS ABAP (1)
Instances are started and stopped using
tools such as the SAP Management Console
(SAP MC), SAP Microsoft Management
Console (SAP MMC), or sapcontrol.
The tools for starting and stopping
communicate with the sapstartsrv process,
which also functions as a Web service.
The tools can be used to start or stop
individual instances or the entire system.
11PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Fundamentals of the SAP Start ServiceStart and Stop process of SAP NetWeaver AS ABAP (2)
The sapstartsrv processes write a control file
to the $(DIR_GLOBAL)\sapcontrol directory
every minute. The name of the control file
indicates the port and host on which the
process can be accessed, and the priority of
its instance.
Using the control files, the sapstartsrv process
recognizes which other sapstartsrv processes
are still active, and starts the related instances
according to their priority.
12PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Fundamentals of the SAP Start ServiceMonitoring tools
The following monitoring tools use the
Web service interface:
SAP Microsoft Management Console (SAP
MMC)
SAP Management Console (SAP MC)
sapcontrol: with sapcontrol (process name on
Windows sapcontrol.exe) you can call the
functions of the Web service interface from the
command field.
13PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Fundamentals of the SAP Start ServiceSAP Microsoft Management Console (SAP MMC)
The SAP snap-in for the Microsoft
Management Console (MMC) provides a
graphical user interface to administer SAP
systems from Windows PCs.
By using SAP MMC you can start, stop, and
restart the instance, display the process list,
and display the ABAP work process table. A
lot of other functions are available as well for
all systems that appear in the tree.
14PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Fundamentals of the SAP Start ServiceSAP Management Console (SAP MC) (1)
SAP Management Console (SAP MC) is an
operating-system-independent application.
This allows you to display various information
on monitoring and perform administration
tasks such as start and stop operations in the
same interface on different operating systems.
The tool is a standalone Java application,
which can be started either as an applet or
locally. The SAP MC is supplied as standard
with the kernel and is ready for use without
any additional installation. All you need is a
Web browser.
15PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Fundamentals of the SAP Start ServiceSAP Management Console (SAP MC) (2)
To start the SAP MC after installing the
relevant SAP NetWeaver AS, enter the
following URL in your browser:
http://<host name>:5<instancenumber>13
https://<host name>:5<instancenumber>14
16PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Fundamentals of the SAP Start ServiceSAPControl
SAPControl is part of the kernel, and is
available on all operating systems. Calling
sapcontrol, without any other options,
provides the syntax description.
SAPControl allows the Web services of the
sapstartsrv processes to be addressed.
Such methods are, for example: Start, Stop,
RestartInstance to operate, GetProcessList
to display the process list of an SAP
instance, StartService, StopService or
RestartService to operate the SAP start
service itself, and many more.
No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP SE or an SAP affiliate company.
The information contained herein may be changed without prior notice. Some software products marketed by SAP SE and its distributors contain proprietary software components
of other software vendors. National product specifications may vary.
These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or warranty of any kind, and SAP or its affiliated
companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP or SAP affiliate company products and services are those that are
set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty.
In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or any related presentation, or to develop or release
any functionality mentioned therein. This document, or any related presentation, and SAP SE’s or its affiliated companies’ strategy and possible future developments, products,
and/or platform directions and functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time for any reason without notice. The
information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. All forward-looking statements are subject to various
risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements,
and they should not be relied upon in making purchasing decisions.
SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE (or an SAP affiliate company)
in Germany and other countries. All other product and service names mentioned are the trademarks of their respective companies.
See http://global.sap.com/corporate-en/legal/copyright/index.epx for additional trademark information and notices.
© 2017 SAP SE or an SAP affiliate company. All rights reserved.
Week 1: Startup, Dispatcher, and Enqueue Services
Unit 2: Common SAP Startup Issues
2PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Common SAP Startup IssuesHow to identify?
Identification:
SAP GUI: cannot log on to system, error is
partner ‘hostname:sapdp<nr>’ not reached
Management console: system is down or partially up
sapcontrol ... -function GetProcessList: displays
RED, GRAY or YELLOW processes
3PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Common SAP Startup IssuesCategories
Startup issues – main categories:
SAP start service not running / cannot start
SAP instance cannot start
4PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Common SAP Startup IssuesSAP start service (1)
SAP start service is normally started automatically at
operating system boot. Exactly one SAP start service
exists for every SAP instance. See also Unit 1.
Typical issues:
on Windows: service doesn't exist
sapstartsrv(.exe) doesn’t exist / corrupt / has zero size
sapstartsrv(.exe) and disp+work(.exe) release / patch is
different
profile doesn't exist / has incorrect entries
authorization issues
localhost name resolution is incorrect
on UNIX: socket or lock file doesn't exist / cannot be created
or accessed
on UNIX: sapservices file doesn’t exist / has incorrect entries
5PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Common SAP Startup IssuesSAP start service (2)
on Windows: service doesn't exist
When opening Windows service manager
(services.msc) the SAP start service
SAP<SID>_<instancenumber> doesn’t appear there.
In this case, you can reinstall it by right-clicking
$(DIR_EXECUTABLE)\sapstartsrv.exe > “Run as
administrator” and specifying:
SAP System ID (SID)
Instance number (NR)
Profile, the instance profile, with Start_Program_xx entries
User, which is <domain>\SAPService<SID>
Password of above user
6PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Common SAP Startup IssuesSAP start service (3)
sapstartsrv(.exe) doesn’t exist / corrupt / has zero size
Check if sapstartsrv(.exe) exists and its size:
on Windows: by file explorer
$(DIR_EXECUTABLE)\sapstartsrv.exe
on UNIX: by ‘ls -al $(DIR_EXECUTABLE)/sapstartsrv’
Validate sapstartsrv(.exe):
on Windows: by double-clicking it, a dialog window should appear
on UNIX: by calling ‘file $(DIR_EXECUTABLE)/sapstartsrv’ the
type ‘executable’ should appear in output; bit version (32 | 64)
and operating system should also match the host
If any of the above fails:
copy sapstartsrv from $(DIR_CT_RUN) to $(DIR_EXECUTABLE)
or extract it from corresponding SAPEXE.SAR or dw.sar archive
7PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Common SAP Startup IssuesSAP start service (4)
sapstartsrv(.exe) and disp+work(.exe) release /
patch is different
They must be on the same level, otherwise false
information may be delivered by sapstartsrv to startup or
monitoring tools.
To compare the versions:
on Windows: right-click
$(DIR_EXECUTABLE)\sapstartsrv.exe Properties
Details tab Product version
same for $(DIR_EXECUTABLE)\disp+work.exe
on UNIX:
‘strings $(DIR_EXECUTABLE)/sapstartsrv | grep SAPProduct’
same for $(DIR_EXECUTABLE)/disp+work
on both: by ‘sapcontrol ... -fuction GetVersionInfo’
8PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Common SAP Startup IssuesSAP start service (5)
profile doesn't exist / has incorrect entries
Check if profile exists:
on Windows: by file explorer
on UNIX: by “ls –al $(DIR_PROFILE)/<profile>”
Check the content:
with a text editor to see whether it is readable, no strange
entries allowed ($ _ : % = / # are valid characters)
whether SAPSYSTEMNAME (SID, e.g. TSL)
SAPSYSTEM (instancenumber, e.g. 00)
INSTANCE_NAME (instance, e.g. DVEBMGS00)
appear in the profile and match the instance of SAP
system
9PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Common SAP Startup IssuesSAP start service (6)
authorization issues
On Windows:
check if SAP_<SID>_*Admin groups have “Read&Execute”
rights on sapstartsrv.exe
<sid>adm and SAPService<SID> users are part of these
groups
On UNIX:
check if sapstartsrv is owned by <sid>adm with execute
rights
10PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Common SAP Startup IssuesSAP start service (7)
localhost name resolution is incorrect
sapcontrol connects to localhost by default to control or
monitor sapstartsrv or SAP instance. To check the name
resolution, run:
niping –v –H localhost
The above must resolve:
localhost to 127.0.0.1 or a valid local IP of the server, and vice-
versa
the full qualify hostname (localhost.<domain>) must be the same
Correct DNS or hosts file if necessary.
See also:
SAP Note 1476386 – Wrong localhost name resolution setting
SAP Note 1872990 – Various error messages related to incorrect
resolution of localhost
11PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Common SAP Startup IssuesSAP start service (8)
on UNIX: socket or lock file doesn't exist /
cannot be created or accessed
When starting sapstartsrv, it needs to create...
/tmp/.sapstream5<instancenumber>13 (HTTP) socket
/tmp/.sapstream5<instancenumber>14 (HTTPS) socket
... or access
/tmp/.sapstartsrv<instancenumber>_sapstartsrv.log
lock file
Check if
they are owned by <sid>adm user
the access rights are correct (rwx for sockets, rw for
lock)
/tmp has the sticky-bit set (rwt)
12PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Common SAP Startup IssuesSAP start service (9)
on UNIX: sapservices doesn’t exist / has incorrect entries
Every local SAP start service is started automatically at
operating system boot as per the definition in
/usr/sap/sapservices.
sapcontrol also uses sapservices file to start or stop the SAP
start service of instances.
Check if:
file exists
can be read by <sid>adm (required for sapcontrol operations) and by
root (required for sapinit operations at boot)
library definition, sapstartsrv, and profile path are correct and exist
profile path must be /usr/sap/<SID>/SYS/profile/<instanceprofile>
13PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Common SAP Startup IssuesTroubleshooting
Parameters / ways of troubleshooting
service/trace = 0 ... 3 in instance profile
default is 0, with this little information is written, only some headers,
list of ports, instance start and stop entries
It is worth setting it to 1 and restarting service to have more details in
$(DIR_HOME)\sapstartsrv.log
Windows: check Event Viewer Windows Logs System entries
regarding SAP start service
UNIX: start service online from command line (without -D option) to
display further information, i.e.
/<path>/sapstartsrv pf=/<path>/instanceprofile
sapcontrol ‘-debug’ option
sapcontrol -debug -prot <prot> -nr <instnr> -function <webmethod>
See also:
SAP Startup Troubleshooting Guide for NW Application Server
14PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Common SAP Startup IssuesStart service running
At the end, check if:
SAP start service is running
sapcontrol calls are working (no error messages appear), e.g.:
sapcontrol -nr <instancenumber> -function GetProcessList
sapcontrol -nr <instancenumber> -function AccessCheck Start
15PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Common SAP Startup IssuesSAP instance (1)
Startup tools
SAP Microsoft Management Console (SAP
MMC)
SAP Management Console (SAP MC)
sapcontrol
startsap script (available only for UNIX,
already deprecated)
See also Unit 1.
16PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Common SAP Startup IssuesSAP instance (2)
SAP instance cannot start
Typical issues:
sapcontrol authorization issues
an instance hasn’t started (yet) that others depend on
missing startup entries in profile / wrong profile used
port is in use
17PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Common SAP Startup IssuesSAP instance (3)
sapcontrol authorization issues
Typical error:
HTTP/1.1 401 Unauthorized
To reproduce:
sapcontrol -prot <prot> -nr <instnr> -function <webmethod>
sapcontrol -prot <prot> -nr <instnr> -function AccessChek
<webmethod>
The Web method is usually Start or Stop in such cases.
Check:
service/protectedwebmethods =
ALL|NONE|DEFAULT|SDEFAULT
It should be set to SDEFAULT (to its default value)
See also
927637 – Web service authentication in sapstartsrv as of
Release 7.00
18PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Common SAP Startup IssuesSAP instance (4)
Correct startup sequence of SAP ABAP system
1. Database
2. Enqueue replication server (ERS) instance(s)
3. ABAP Central Services (ASCS) Instance, or if no ASCS
exists, the instance which includes message and enqueue
services: the classic central instance (CI)
4. Primary application server (PAS) instance and additional
application server (AAS) instance(s), or dialog instance(s)
(DI)
19PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Common SAP Startup IssuesSAP instance (5)
an instance hasn’t started (yet) that others depend on
The corresponding error is, for example:
in dev_disp
*** ERROR => DpTriggerMsAttach: Attach to Message Server
failed in dev_wXX trace
*** ERROR => EncNiConnect: unable to connect
(NIECONN_REFUSED)
Check the status of instances by:
sapcontrol –prot <protocol> -nr <instancenr>
GetSystemInstanceList
sapcontrol –prot <protocol> -host <host> -nr <instnr>
GetprocessList
The status of the instance that others depend on must be
‘GREEN’ before starting further instances; you can check
this with the above sapcontrol calls.
20PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Common SAP Startup IssuesSAP instance (7)
unavailability of startup entries in profile
Check which profile is in use:
on Windows: click Properties of
SAP<SID>_<instancenumber> service
on UNIX: outcome of ’ps –ef | grep sapstartsrv’ will
display /path/profile following pf=
Check startup entries:
open the above profile with text editor and check if
there are Start_Program_xx or Restart_Program_xx
entries available
21PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Common SAP Startup IssuesSAP instance (8)
port is in use
SAP system ports must be free to be able to start the
system. Ports are among others:
dispatcher port (32<instancenumber>)
gateway port (33<instancenumber>)
message server port (36<instancenr>) / internal port (39<instnr>)
ICM port (80<instancenumber>)
Error:
SI_EPORT_INUSE / NIESERV_USED
Use lsof or netstat to find what is blocking a port.
See also
TCP/IP Ports of All SAP Products
1845121 – Address already in use – how to find the process
listening on port ...
22PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Common SAP Startup IssuesTroubleshooting
Tips for troubleshooting
Check following:
newest developer traces in work folder of affected instance
entries of sapstart(srv).log or stderr<xx> in work folder
instance status by sapcontrol calls GetSystemInstanceList,
GetProcessList, AccessCheck <webmethod>
Parameters:
rdisp/TRACE = 0...3
Default value is 1
The higher the level, the more information will be written in
traces
See also:
SAP Startup Troubleshooting Guide for NetWeaver
Application Server
Standard error files of SAP workprocesses during
startup
Trace files of the startup processes – located in
/home/<SID>adm directory
Startup log of database, dispatcher and message
server
Standard trace files of the message server
Standard trace files of the dispatcher
Standard trace files of SAP workprocesses
No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP SE or an SAP affiliate company.
The information contained herein may be changed without prior notice. Some software products marketed by SAP SE and its distributors contain proprietary software components
of other software vendors. National product specifications may vary.
These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or warranty of any kind, and SAP or its affiliated
companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP or SAP affiliate company products and services are those that are
set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty.
In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or any related presentation, or to develop or release
any functionality mentioned therein. This document, or any related presentation, and SAP SE’s or its affiliated companies’ strategy and possible future developments, products,
and/or platform directions and functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time for any reason without notice. The
information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. All forward-looking statements are subject to various
risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements,
and they should not be relied upon in making purchasing decisions.
SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE (or an SAP affiliate company)
in Germany and other countries. All other product and service names mentioned are the trademarks of their respective companies.
See http://global.sap.com/corporate-en/legal/copyright/index.epx for additional trademark information and notices.
© 2017 SAP SE or an SAP affiliate company. All rights reserved.
Week 1: Startup, Dispatcher, and Enqueue Services
Unit 3: Introduction to Dispatcher Queues
2PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Introduction to Dispatcher QueuesOld concept prior to SAP NetWeaver 7.40
The dispatcher
has a queue for each work process type
Is responsible for the state of each work process
has to find a free work process for dispatching
Each work process has a local queue
for processing requests directly (without communication
via the dispatcher)
for processing asynchronous request (e.g., an aRFC
response)
The communication partner has to know whether a
session is attached to a work process or whether it
has to send the request via the dispatcher
3PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Introduction to Dispatcher Queues New concept
A flexible request queue mechanism:
Queues can be created/destroyed dynamically
Multiple processes can listen to the same queue
A process can acquire a queue for exclusive use
Processes can attach/detach to the queue infrastructure dynamically
How do you use the new request queues?
Create one queue for each session
Work processes listen to different queues and take new requests
when they are free
Sessions can specify which kind of requests they can currently
process
Communication partners do not have to take care of whether a
session is active in a work process or not
4PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Introduction to Dispatcher Queues Request queues overview
Processes have their own queue
Dispatcher / Gateway / ICM
Work process queues (each of which is handled by the
dedicated work process)
Session request queues
Each session has its own queue (handled by any free
work process of proper type)
Embryo queues: 1 for each worker type; sessions are
only created when requests are processed
Dispatch queues
When a session queue is not attached to a work
process, it is added to a “dispatch queue”.
There are 3, one for each session priority: high, medium,
low.
5PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Introduction to Dispatcher Queues Monitoring of request queues (1)
Request queues of server processes
SM51: Goto Information Queue Information
Max. no of requests = 7 x rdisp/elem_per_queue
Local Work Processes
6PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Introduction to Dispatcher Queues Monitoring of request queues (2)
Work process request queue
SM50: Process Information (CTRL-SHIFT-F11)
Process Information
Session request queues and session
request mask
SM04: User Technical Information
7PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Introduction to Dispatcher Queues Request dispatching
Finding session queues for processing:
Each work process has to find the session queues that
need to be processed.
Request processing should be “fair”:
– Requests for sessions of the same kind are expected to
be handled in order of arrival.
– Requests for some sessions should have higher or lower
priority (GUI or batch processing).
Dispatch queues (queues of session queues):
Are observed by the work processes
There are three such queues for three priorities: high,
medium, low
8PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Introduction to Dispatcher Queues Priorities as of 7.4x
High: request from UI (by default only SAP GUI)
Medium: all other requests (HTTP, RFC)
Low: batch processing (RFC started from BTC session)
Child processes have same or lower priority as their mother
process ABAP API to set RFC/HTTP session to high priority
(by default the priority is normal)
Runtime limit for dialog request:
rdisp/scheduler/prio_high/max_runtime Default: 10 min
rdisp/scheduler/prio_normal/max_runtime Default: 60 min
rdisp/scheduler/prio_low/max_runtime Default: no limit
9PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Introduction to Dispatcher Queues Configuration changes
High load quota:
Protect the request queue from being flooded by requests for a certain worker type
Configuration via profile parameter rdisp/high_load_quota Default: load=90, queue=5
Task limit of sessions:
Protect the request queue from being flooded by requests for a single session
Configuration via profile parameter rdisp/task_limit Default: 1000
SAP Note: 2001276 – Changed configuration as of 7.40 SP2
No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP SE or an SAP affiliate company.
The information contained herein may be changed without prior notice. Some software products marketed by SAP SE and its distributors contain proprietary software components
of other software vendors. National product specifications may vary.
These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or warranty of any kind, and SAP or its affiliated
companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP or SAP affiliate company products and services are those that are
set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty.
In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or any related presentation, or to develop or release
any functionality mentioned therein. This document, or any related presentation, and SAP SE’s or its affiliated companies’ strategy and possible future developments, products,
and/or platform directions and functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time for any reason without notice. The
information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. All forward-looking statements are subject to various
risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements,
and they should not be relied upon in making purchasing decisions.
SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE (or an SAP affiliate company)
in Germany and other countries. All other product and service names mentioned are the trademarks of their respective companies.
See http://global.sap.com/corporate-en/legal/copyright/index.epx for additional trademark information and notices.
© 2017 SAP SE or an SAP affiliate company. All rights reserved.
Week 1: Startup, Dispatcher, and Enqueue Services
Unit 4: Tools for Troubleshooting
2PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Tools for TroubleshootingSnapshots
Snapshots can be triggered manually or are written by the
dispatcher when severe errors occurs in the instance like:
hard shutdown
exhausted resources
deadlock handling
icm/gateway crash
They are written in several steps:
The dispatcher writes all the accessible information to its trace file
All running work processes write information about what they are
currently executing (C stack)
A subsection of the trace files is written to the file system
Snapshots are packed as zip archives in the directory
$(DIR_GLOBAL)/sapcontrol/snapshots
3PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Tools for TroubleshootingC stack (1)
Can be obtained:
Via transaction SM50
Externally via executable “sapstack”
sapstack <pid>
Unix: sapstack –tree <pid>
Windows: Sapstack Global\SAP<SID>_<NR>
SAP Note 1964673 – C-Call stack analysis
sapcontrol -nr <instance_number> -function
GetCallstack <pid>
4PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Tools for TroubleshootingC stack (2)
SAP MMC, right-click a process:
Below the <signal
handler> we have
the running
functions
5PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Tools for TroubleshootingSemaphore monitoring (1)
If a work process locks a particular resource,
it sets a semaphore.
Another work process that also wants to access
it must then wait.
In this case, the number of the semaphore is
displayed by SM50.
By pressing F1 in the Semaphore field, all the
SAP-related semaphores appear.
The wiki page What do the semaphores mean?
list all the semaphores dependent of the kernel
release.
6PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Tools for TroubleshootingSemaphore monitoring (2)
SAP Note 2027885 – External semaphore monitoring
semwatch pf=<profile>
Checks all possible SAP semaphores on all the instances
in one host.
Creates C stack of processes holding a semaphore longer
than 1s.
Can execute any user-defined command.
Can kill processes after a maximum semaphore hold time.
Independent of SAP release (since release 6.10)
7PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Tools for TroubleshootingSAPControl
The WebSevices executed via SAPControl to monitor
the system should be restricted with the SAP profile
parameter “service/protectedwebmethods”.
sapcontrol -nr <nr> -function GetProcessList
sapcontrol -nr 00 -function ABAPGetWPTable
sapcontrol -nr 00 -function GetQueueStatistic [-repeat -1 1]
See Week 1 – Unit 1
8PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Tools for Troubleshooting/sdf/mon (1)
Collect snapshots of monitoring transactions and
store this information for later analysis
Via transaction
/sdf/mon Schedule New Monitoring
9PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Tools for Troubleshooting/sdf/mon (2)
No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP SE or an SAP affiliate company.
The information contained herein may be changed without prior notice. Some software products marketed by SAP SE and its distributors contain proprietary software components
of other software vendors. National product specifications may vary.
These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or warranty of any kind, and SAP or its affiliated
companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP or SAP affiliate company products and services are those that are
set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty.
In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or any related presentation, or to develop or release
any functionality mentioned therein. This document, or any related presentation, and SAP SE’s or its affiliated companies’ strategy and possible future developments, products,
and/or platform directions and functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time for any reason without notice. The
information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. All forward-looking statements are subject to various
risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements,
and they should not be relied upon in making purchasing decisions.
SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE (or an SAP affiliate company)
in Germany and other countries. All other product and service names mentioned are the trademarks of their respective companies.
See http://global.sap.com/corporate-en/legal/copyright/index.epx for additional trademark information and notices.
© 2017 SAP SE or an SAP affiliate company. All rights reserved.
Week 1: Startup, Dispatcher, and Enqueue Services
Unit 5: Introduction to the Enqueue Service
2PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Introduction to the Enqueue ServerEnqueue functionality (1)
Locking of application objects (DB entries) against
concurrent access:
The application cannot react to DB locks but could get
blocked
The Enqueue server allows locking on application
level, independent from the database locks to keep
data consistency
The knowledge about the enqueue locks must be
within a single instance: the “enqueue server”
3PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Introduction to the Enqueue ServerEnqueue functionality (2)
SAP logical unit of work (LUW)
Dialog part of transaction update part
DB transaction DB transaction DB transaction
DIA-Wp 1 DIA-Wp 2 DIA-Wp 3
COMMIT
WORK
DB transaction
Update
Update-Wp
4PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Introduction to the Enqueue ServerEnqueue parts
Parts of the enqueue logic
Lock argument
Lock owners (dialog & update)
Backup flag
Cumulation counter
Pure informational parts
Client
User name
Date, time, work process number, host
name (encoded in enqueue owner)
Transaction code
Lock object name
5PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Introduction to the Enqueue ServerWhy standalone enqueue?
High availability requirements of customers
cannot be fulfilled with old enqueue architecture
Enqueue table in main memory
lost in case of a crash
single point of failure (SPOF)
Demands
– eliminate all SPOFs
– based on standard technologies
– as few changes to enqueue library as possible
Idea: create in memory replica on backup host
– in memory: due to performance requirements
6PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Introduction to the Enqueue ServerCommunication architecture
Standalone
Enqueue
Server
Replication
Server
Replication
TableEnqueue Table
Enqueue
Client
Enqueue
ClientEnqueue
Client
Enqueue
Client Enqueue
Client
TCP/IP
7PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Introduction to the Enqueue ServerThe Enqueue table
To check the location of the
enqueue server
Area of (shared) memory that holds the enqueues; three
sub-tables:
Owner tables
Argument tables
Lock table
Table size defined by parameter enque/table_size
8PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Introduction to the Enqueue ServerEnqueue processing
Worker Thread
Replication
Thread
I/O-Thread
I/O-Thread
Enqueue
Library Enqueue
Table
Listen-Thread
Accept new
connections
Repl.
functionality
Replication
server
TCP/IPResponse Store
in shared
Memory
Enqueue
request
Enqueue
response
Rep.
request
Rep.
response
Enqueue
response
Thread
control
Enqueue
client
9PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Introduction to the Enqueue ServerConfiguration
Enqueue Client
enque/process_location = REMOTESA
enque/serverhost = <hostname of enque server>
enque/serverinst = <instance number>
enque/encni/port = default: sapdp$(enque/serverinst)
enque/con_retries * enque/con_timeout:
Enqueue Server
enque/server/max_clients
enque/server/threadcount = <number of I/O threads>
enque/server/replication = [true|false]
enque/encni/repl_port
enque/table_size
10PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Introduction to the Enqueue ServerGood to know
Lock object (SE11) Lock/Unlock function
modules
Form routine in function group SENA
SAPLSENA
Displaying trace files:
SAP Microsoft Management Console: Windows
SAP Management Console
File ENQHISOVR (work directory) gives
information about the top usages when an
overflow of the enqueue table occurs
SAP Note 746138 – Analyzing lock table
overflows
No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP SE or an SAP affiliate company.
The information contained herein may be changed without prior notice. Some software products marketed by SAP SE and its distributors contain proprietary software components
of other software vendors. National product specifications may vary.
These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or warranty of any kind, and SAP or its affiliated
companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP or SAP affiliate company products and services are those that are
set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty.
In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or any related presentation, or to develop or release
any functionality mentioned therein. This document, or any related presentation, and SAP SE’s or its affiliated companies’ strategy and possible future developments, products,
and/or platform directions and functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time for any reason without notice. The
information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. All forward-looking statements are subject to various
risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements,
and they should not be relied upon in making purchasing decisions.
SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE (or an SAP affiliate company)
in Germany and other countries. All other product and service names mentioned are the trademarks of their respective companies.
See http://global.sap.com/corporate-en/legal/copyright/index.epx for additional trademark information and notices.
© 2017 SAP SE or an SAP affiliate company. All rights reserved.
Week 1: Startup, Dispatcher, and Enqueue Services
Unit 6: Enqueue Performance
2PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Enqueue PerformanceAspects of enqueue performance
Enqueue calls consist of a communication part and a processing part
Identify if the problem is a communication problem or a processing problem
Do you have performance problems on both sides or only in the DI side?
Both sides: enqueue processing issue
Only on DI: communication issue
Enqueue
Server
Dialog
Instance
Communication channel
standalone enqueue server: TCP/IP
3PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Enqueue PerformanceGeneral recommendations
The standalone enqueue server (ENSA) and database
shouldn't be placed in the same host sharing CPU
resources.
The number of I/O threads used in the enqueue server
(parameter enque/server/threadcount) should be 4.
A backup file is not needed when running standalone
enqueue with replication server.
Memory shortages can cause dramatic performance
decrease and replication interruption.
SAP Note 920979 – Out of memory in the standalone
enqueue server.
4PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Enqueue PerformanceSymptoms (1)
STAD
Check enqueue time and count in STAD
In this example, we see 13 enqueue calls with a total time of 81ms 6.2ms per call
5PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Enqueue PerformanceSymptoms (2)
SM12
Check for big number of enqueue collisions
Check for operations that are very costly (read/backup) if
they occur very often (compared to the other operations)
Check that Wait time is smaller than Time in Lock Server
6PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Enqueue PerformanceSymptoms (3)
SM50/SM66
SAPLSENA running for long time in several
work processes
Possible reasons:
Disconnection with the enqueue server
Enqueue handling by the application
7PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
On the client side (dev_w*, dev_disp) On the server side (dev_enq*)
Enqueue PerformanceTools: Developer trace files
8PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Enqueue PerformanceTools: Enqueue logging
Which locks were set and how long were they
retained?
In which order were the lock operations issued?
What is the result of an operation? Was it
successful?
Were there any lock conflicts and, if so, which
lock and users were concerned?
How many operations were carried out? What is
the current throughput?
How fast are locks transferred to the ERS?
Which enqueue client carried out a particular
operation?
SAP Note 2126913 – ENQU: The enqueue log
9PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Enqueue PerformanceTools: Z_ENQUEUE_PERF
Functions:
Checks the configuration of the enqueue service
Performs a range of performance tests to assess possible
bottlenecks
Performs a range of tests to detect known problems
SAP Note 1320810 – Z_ENQUEUE_PERF
Via SM12; the program is called RSMONENQ_PERF
10PUBLIC© 2017 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Enqueue PerformanceTools: enqt
Command Line Enqueue Test Tool
Can act as enqueue client and can attach to the
productive enqueue table for monitoring
can do almost everything like the SM12 test tools
used for long-term monitoring of the enqueue
performance (opcode 8)
could be started with a client profile or with the
enqueue server profile, depending on what you want
No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP SE or an SAP affiliate company.
The information contained herein may be changed without prior notice. Some software products marketed by SAP SE and its distributors contain proprietary software components
of other software vendors. National product specifications may vary.
These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or warranty of any kind, and SAP or its affiliated
companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP or SAP affiliate company products and services are those that are
set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty.
In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or any related presentation, or to develop or release
any functionality mentioned therein. This document, or any related presentation, and SAP SE’s or its affiliated companies’ strategy and possible future developments, products,
and/or platform directions and functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time for any reason without notice. The
information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. All forward-looking statements are subject to various
risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements,
and they should not be relied upon in making purchasing decisions.
SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE (or an SAP affiliate company)
in Germany and other countries. All other product and service names mentioned are the trademarks of their respective companies.
See http://global.sap.com/corporate-en/legal/copyright/index.epx for additional trademark information and notices.
© 2017 SAP SE or an SAP affiliate company. All rights reserved.