EngageOne Deliver Performance White Paper Page 1 of 40
Performance Benchmark White Paper
EngageOne Deliver
Performance Benchmarks
EngageOne Deliver 2.5
June 2018
©2018 Pitney Bowe Software, Inc.
EngageOne Deliver Performance White Paper Page 2 of 40
Performance Benchmark White Paper
Contents Purpose & Scope ..................................................................................................................................................... 4
Architecture ............................................................................................................................................................ 4
Test Environment and Methodology ...................................................................................................................... 5
Hardware Specifications ..................................................................................................................................... 6
Testing Scenarios ................................................................................................................................................ 7
Email Performance Testing ..................................................................................................................................... 8
Email Test Scenarios (Batch processing) ............................................................................................................. 9
Email Performance for Different Batch Sizes .................................................................................................... 10
Email Performance for Different Attachment Sizes .......................................................................................... 11
Email Performance for Optional Features Turned ON ...................................................................................... 12
SMS Performance Testing (Batch Processing) ...................................................................................................... 13
SMS Test Scenarios ........................................................................................................................................... 13
SMS Performance for Different Batch Sizes ...................................................................................................... 14
SMS Performance with Archiving ON and OFF ................................................................................................. 15
REST API Testing ................................................................................................................................................... 16
Email Test Scenarios (REST API) ........................................................................................................................ 16
SMS Test Scenarios (REST API) .......................................................................................................................... 18
Inbound SMS Testing ............................................................................................................................................ 20
Inbound SMS Performance ............................................................................................................................... 21
Bounce Email Testing ............................................................................................................................................ 22
Bounce Email Performance ............................................................................................................................... 23
Inbound New Email Testing .................................................................................................................................. 24
Inbound New Email Performance ..................................................................................................................... 25
Mobile Push Notification Testing ......................................................................................................................... 26
Mobile Push Notification Performance ............................................................................................................ 27
System Load Results ............................................................................................................................................. 28
Scenario 1: Email, Async mode, 5 files, 5 threads ............................................................................................. 29
EngageOne Deliver Performance White Paper Page 3 of 40
Performance Benchmark White Paper
Scenario 2: Email, Async mode, 5 files, 5 threads, 120 KB attachment ............................................................ 31
Scenario 3: Email, Async mode, 5 files, 5 threads, 120 KB attachment, Digital Signature ............................... 32
Scenario 4: Email, Async mode, 5 files, 5 threads, 1 MB attachment .............................................................. 33
Scenario 5: SMS, 10 files, 5 threads .................................................................................................................. 34
Scenario 6: SMS, 10 files, 5 threads, Archival ON ............................................................................................. 36
EngageOne Deliver Performance Recommendations .......................................................................................... 37
Configuration considerations for optimal performance ...................................................................................... 39
Running email outbound processing in Async mode ........................................................................................ 39
Settings for parallel profile threads and gateway connections ........................................................................ 40
EngageOne Deliver Performance White Paper Page 4 of 40
Performance Benchmark White Paper
Purpose & Scope The purpose of this document is to provide performance results achieved after testing the EngageOne Deliver application in different user scenarios and test data. These results can be used by the customers to estimate their hardware and software requirements based on their organizational data to be processed.
Architecture The following diagram shows the EngageOne Deliver physical architecture.
EngageOne Deliver Performance White Paper Page 5 of 40
Performance Benchmark White Paper
Test Environment and Methodology The following sections of this document describe performance benchmarks for EngageOne Deliver running under Windows with a Tomcat server. Test results are available for both Batch (DIJ) processing as well as REST APIs. This performance benchmark includes testing for Email & SMS using outbound and inbound scenarios.
The testing environment consisted of three dedicated servers configured as follows:
• EngageOne Deliver application server (Tomcat) • Database Server (SQL Server) • Mail server (hMailServer) and SMS gateway (SMPP simulator).
The list below summarizes key information related to the environment and its configuration:-
• Latency between servers was around 1 to 3 ms.
• Before starting the test scenarios, the main table for outbound and inbound had around 5 million
and 1.5 million records respectively.
• The default setting was used for async.properties.
• In the outboundProcessor.properties file, the outbound.email.messageCountPerConnection=19
was set to 100
• The Vendor folder was kept local to Tomcat server machine.
• For REST API’s, JMeter /SOAPUI was deployed on the email server machine.
• Performance testing was carried out on a single node deployment of Deliver. However, Deliver can
be horizontally scaled using a multi-node clustering deployment.
• Email DIJ based processing can be run in sync/async mode. In async mode, email composition and
sending process becomes decoupled. As a result, a higher sending (email gateway and network
bandwidth) capacity can be utilized, if available. This may provide higher throughput although
fewer file processing threads in the outbound process were used. REST, by default runs in async
mode. SMS does not have an async mode as composition time in SMS is almost negligible.
• DIJ based performance tests were run with 1 processing thread and 5 processing threads
respectively, hence 5 input files were used for each test as a standard for most scenarios.
EngageOne Deliver Performance White Paper Page 6 of 40
Performance Benchmark White Paper
Hardware Specifications This section describes the hardware specifications of the machine(s) used for performance testing.
EngageOne Deliver Server
Operating System Windows Server 2012 R2 CPU Intel(R) Xeon(R) CPU E7-4820 v3 @ 1.90 GHz 1.90 GHz (4 Processor) Memory 8 GB Disk Drives 80 GB JVM Memory (Tomcat) 2 GB Apache Tomcat version Apache Tomcat 8.5.16 Java version Java 8
Database Server
Operating System Windows Server 2016 CPU Intel(R) Xeon(R) CPU X742 @ 2.67 GHz, 2.67 GHz (2 Processor) Memory 8 GB Disk Drives 80 GB SQL Server version SQL Server 2016
Mail / SMS Server & JMeter (for REST)
Operating System Windows Server 2012 R2 CPU Intel(R) Xeon(R) CPU X742 @ 2.67 GHz, 2.67 GHz (2 Processor) Memory 12 GB Disk Drives 120 GB Mail server version hMailServer 5.6.6
EngageOne Deliver Performance White Paper Page 7 of 40
Performance Benchmark White Paper
Testing Scenarios The following test scenarios were performed and tracked:
• Baseline functionality to provide baseline performance numbers • Benchmark tests to determine the impact on performance using different options • Customer(s) specific scenarios derived from the field • Load tests to check the threshold limit
EngageOne Deliver Performance White Paper Page 8 of 40
Performance Benchmark White Paper
Email Performance Testing This section presents performance testing results for email processing. DIJ based Batch Email processing can be run in both the legacy sync(multithreading) and the newly introduced async modes. Async mode allows for the sending process to be run independently of the email composition process; this provides flexibility in configuring sending threads & gateway connections. Async mode is the default and preferred option as it provides improved performance in most scenarios where higher network bandwidth/speed and email server capacity is available.
EngageOne Deliver Performance White Paper Page 9 of 40
Performance Benchmark White Paper
Email Test Scenarios (Batch processing) The following scenarios were executed to test the performance of email processing. For Async mode 10 sending threads were set up which effectively means 10 simultaneous connections to the email gateway.
Table 1: Email Test Results for Sync and Async mode
S. No. Number of Input
Files
Number of
Records / Input
File
Attachment Size/Type1
Feature Enabled
Messages Delivered/Sec with 1 thread
Messages Delivered/Sec with 5 threads
Sync Async Sync Async
1 5 2000 No Attachment NA 23 41 39 51
2 5 2000 120KB / (Personalized) NA 14 39 36 44
3 5 2000 120KB / (Personalized)
Digital Signature 9 31 31 34
4 5 2000 120KB / (Personalized) Archive 13 35 33 36
5 5 2000 1 MB / (Personalized) NA 5 17 18 36
6 1 2000 120KB / (Personalized) NA - - 20 37
7 3 2000 120KB / (Personalized) NA - - 28 42
8 5 50000 No Attachment NA 20 36 30 40
9 5 20000 120KB / (Personalized) NA 12 29 28 36
10 5 10000 1 MB / (Personalized) NA 4 14 15 27
1 Note that a personalized attachment type indicates that the attachment is personalized for the recipient of the Email
EngageOne Deliver Performance White Paper Page 10 of 40
Performance Benchmark White Paper
Email Performance for Different Batch Sizes This section presents performance results for a number of test cases that represent actual customer usage. The following graph shows performance numbers for test cases where there are a varying number of input DIJ files.
Graph 1: Email Performance for Different Batch Sizes
0
10
20
30
40
50
60
1 File of 2K records withAttachment
3 Files of 2K records eachwith Attachment
5 Files of 2K records eachwith Attachment
5 Files of 2K records eachwithout Attachment
Mes
sage
s per
sec
Email Performance (5 threads)
Sync Async
EngageOne Deliver Performance White Paper Page 11 of 40
Performance Benchmark White Paper
Email Performance for Different Attachment Sizes The following graph indicates the impact of increasing attachments size on performance.
Graph 2: Email Performance for Increasing Attachment Size
0
10
20
30
40
50
60
1 Thread 5 Thread 1 Thread 5 Thread
Mes
sage
s per
sec
Email Performance - Increasing Attachment Size
No Attachment 120KB / (Personalized) 1 MB / (Personalized)
Sync Mode Async Mode
EngageOne Deliver Performance White Paper Page 12 of 40
Performance Benchmark White Paper
Email Performance for Optional Features Turned ON EngageOne Deliver provides a number of optional features, which can be turned ON or OFF. This section explains how performance varies when you turn the features ON one-by-one.
Graph 3: Email Performance with Optional Features ON
0
10
20
30
40
50
60
No Attachment 120KB Attachment Digital Signature Archive 1 MB Attachment
Mes
sage
s per
sec
Email Performance - Optional Features ON
Sync Mode with 1 Thread Async Mode with 1 Thread Sync Mode with 5 Thread Async Mode with 5 Thread
EngageOne Deliver Performance White Paper Page 13 of 40
Performance Benchmark White Paper
SMS Performance Testing (Batch Processing) This section presents performance testing results for SMS processing. Note that SMS runs in sync mode only.
SMS Test Scenarios The following scenarios were executed to test the performance of SMS processing:
Table 2: SMS Test Data
S. No. Number of Input Files
Number of Records / Input
File
Feature Enabled
Messages Delivered/Sec with 1 thread
Messages Delivered/Sec with 5 threads
1 1 5000 NA 34 35
2 1 10000 NA 35 36
3 5 5000 NA 36 71
4 5 10000 NA 35 72
5 10 5000 NA 37 91
6 10 10000 NA 36 90
7 1 5000 Archive 28 34
8 5 5000 Archive 33 65
9 10 5000 Archive 27 74
10 5 50000 Archive 25 62
11 5 50000 NA 32 72
EngageOne Deliver Performance White Paper Page 14 of 40
Performance Benchmark White Paper
SMS Performance for Different Batch Sizes This section presents performance results for a number of test cases that represent actual customer usage. The following graph presents performance numbers for test cases where the number of input DIJ files varies.
Graph 4: SMS Performance for Different Batch Sizes
0
10
20
30
40
50
60
70
80
90
100
5K (1 Input fileof 5K records)
10K (1 Input fileof 10K records)
25K (5 Inputfiles of 5Krecords)
50K (5 Inputfiles of 10K
records)
50K (10 Inputfiles of 5Krecords)
100K (10 Inputfiles of 10K
records)
250K (5 Inputfiles of 50K
records)
Mes
sage
s per
sec
SMS Performance
1 Thread 5 Thread
EngageOne Deliver Performance White Paper Page 15 of 40
Performance Benchmark White Paper
SMS Performance with Archiving ON and OFF This section presents the performance numbers achieved after testing SMS processing where the archive feature is enabled and disabled.
Graph 5: SMS Performance with Archiving ON/OFF
0
10
20
30
40
50
60
70
80
90
100
5K (1 Inputfile of 5Krecords)
25K (5 Inputfiles of 5Krecords)
50K (10 Inputfiles of 5Krecords)
250K (5 Inputfiles of 50K
records)
5K (1 Inputfile of 5Krecords)
25K (5 Inputfiles of 5Krecords)
50K (10 Inputfiles of 5Krecords)
250K (5 Inputfiles of 50K
records)
Mes
sage
s per
sec
SMS Performance - Impact of Archival
1 Thread 5 Thread
Without Archival With Archival
EngageOne Deliver Performance White Paper Page 16 of 40
Performance Benchmark White Paper
REST API Testing This section presents performance testing results for Email and SMS processing through REST APIs. Note that the JMeter tool was used to measure user hits.
Email Test Scenarios (REST API) The following scenarios were executed to test the performance of email processing:
Table 1: Email Test Results for REST APIs
S. No. No. of Concurrent Users
No. of Hits/user
Attachment Size/Type
Feature Enabled
Average TPS (Message Delivered per sec)
Average Response Time (ms)
1 2 5000 No Attachment NA 41 47
2 5 5000 No Attachment NA 34 124
3 10 5000 No Attachment NA 25 323
4 2 5000 120KB / (Personalized) NA 29 54
5 5 5000 120KB / (Personalized) NA 27 141
6 10 5000 120KB / (Personalized) NA 25 291
7 2 5000 500KB / (Personalized) NA 18 78
8 2 5000 1MB / (Personalized) NA 11 124
9 5 5000 No Attachment Digital Sign 26 159
10 5 5000 No Attachment Archival 29 124
11 5 5000 No Attachment Digital Sign, Archival 24 183
12 2 50000 No Attachment NA 27 50
EngageOne Deliver Performance White Paper Page 17 of 40
Performance Benchmark White Paper
REST API – Email Performance with different user scenarios
REST API – Email Performance with Optional Features enabled
05
1015202530354045
2 Users with5K hits each
withoutattachment
5 Users with5K hits each
withoutattachment
10 Userswith 5K hits
each withoutattachment
2 Users with5K hits eachwith 120 Kbattachment
5 Users with5K hits eachwith 120 Kbattachment
10 Userswith 5K hitseach with
120 Kbattachment
2 Users with5K hits eachwith 500 Kbattachment
2 Users with5K hits eachwith 1000 Kbattachment
2 Users with50K hits each
withoutattachment
Mes
sage
s per
Sec
REST API - Email Performance
0
5
10
15
20
25
30
35
40
No Attachment 120KB / (Personalized) Digital Sign Archival Digital Sign, Archival
Mes
sage
s per
sec
REST API Email Performance - Optional Features ON
EngageOne Deliver Performance White Paper Page 18 of 40
Performance Benchmark White Paper
SMS Test Scenarios (REST API) The following scenarios were executed to test the performance of SMS processing:
Table 1: SMS Test Results for REST APIs
S. No. No. of Concurrent Users
No. of Hits/user
Feature Enabled
Average TPS (Message Delivered per sec)
Average Response Time (ms)
1 2 10000 NA 63 33
2 5 10000 NA 60 81
3 10 10000 NA 57 164
4 2 10000 Archival 61 32
5 5 10000 Archival 58 79
6 10 10000 Archival 56 166
7 2 50000 NA 60 33
8 2 50000 Archival 58 34
9 2 100000 NA 61 33
10 2 100000 Archival 59 167
11 5 50000 NA 59 87
12 5 50000 Archival 57 85
EngageOne Deliver Performance White Paper Page 19 of 40
Performance Benchmark White Paper
REST API – SMS Performance with different user scenarios
REST API – SMS Performance showing impact of Archival
54
55
56
57
58
59
60
61
62
63
64
2 Users with 10K hits each 5 Users with 10K hits each 10 Users with 10K hits each 2 Users with 50K hits each
Mes
sage
s per
sec
REST API - SMS Performance
52
54
56
58
60
62
64
2 Users with 10K hits each 5 Users with 10K hits each 10 Users with 10K hits each 2 Users with 50K hits each
Mes
sage
s per
sec
REST API SMS Performance - Impact of Archival
Without Archival With Archival
EngageOne Deliver Performance White Paper Page 20 of 40
Performance Benchmark White Paper
Inbound SMS Testing This section presents performance testing results for Inbound SMS (MO). Tests were run with inbound running in existing sync mode and the newly introduced async mode.
These processing modes are controlled by the enable.InboundSmpp.ThreadPool property in SMPPService.properties file (valid values are true/false). When the value is set to true, async mode is in force. All other properties of Inbound SMS thread pool were kept at default values. Note that Async mode is the default and preferred option.
To simulate fresh MO messages, MO URL of SMPP simulator was hit from JMeter. These MOs were then read by Deliver through the Inbound SMS profile.
Inbound SMS Test Results
S. No. Inbound (sync / async)
Outbound SMS running in parallel
Feature Enabled
Average TPS (Message received per sec)
1 sync false NA 38
2 async false NA 80
3 sync true NA 34
4 async true NA 38
5 sync false Archival 8
6 async false Archival 36
EngageOne Deliver Performance White Paper Page 21 of 40
Performance Benchmark White Paper
Inbound SMS Performance
0102030405060708090
Sync data processing Async data processing Sync data processing Async data processing
Mes
sage
s per
sec
Inbound SMS - Performance
Without Outbound Processing in With Outbound Processing in Parallel
EngageOne Deliver Performance White Paper Page 22 of 40
Performance Benchmark White Paper
Bounce Email Testing This section presents performance testing results for Inbound Bounce email handling. Tests were run with inbound running in existing sync mode and the newly introduced async mode.
Outbound process for sending emails was also running in parallel to inbound bounce process. DIJ based outbound process was used for sending outbound messages.
Bounce Email Test Results
S. No. # DIJ files
# Records /
DIJ file
# Outbound
Threads Email
Attachment
Inbound process type
(Sync / Async)
Average Inbound TPS (Bounce
received per sec)
1 5 2000 5 NA S 5
2 5 2000 5 120 KB S 3
3 5 2000 5 500 KB S 1
4 5 10000 5 NA S 4
5 5 2000 5 NA A 13
6 5 2000 5 120 KB A 6
7 5 2000 5 500 KB A 2
8 5 10000 5 NA A 7
EngageOne Deliver Performance White Paper Page 23 of 40
Performance Benchmark White Paper
Bounce Email Performance
0
2
4
6
8
10
12
14
10K Records 50K Records
Mes
sage
s per
sec
Bounce Performance
Sync Async
0
2
4
6
8
10
12
14
No Attachment 120 KB 500 KB
Mes
sage
s per
sec
Bounce Performance - Impact of Attachment Size
Sync Async
EngageOne Deliver Performance White Paper Page 24 of 40
Performance Benchmark White Paper
Inbound New Email Testing This section presents performance testing results for inbound new email handling. Tests were run with inbound running in existing sync mode and the newly introduced async mode.
New emails were push to hMailServer through JMeter SMTP sampler, which were later read by EngageOne Deliver inbound profile
New Email Test Results
S. No. # JMeter threads
# Records /
thread Email
Attachment
Archival
(Original Parts only)
Inbound process type
(Sync / Async)
Average Inbound TPS (Email received
per sec)
1 5 5000 NA NA S 10
2 5 5000 NA NA A 25
3 5 5000 NA Y S 4
4 5 5000 NA Y A 21
5 5 5000 120 KB NA S 4
6 5 5000 120 KB NA A 9
7 5 5000 120 KB Y S 3
8 5 5000 120 KB Y A 8
EngageOne Deliver Performance White Paper Page 25 of 40
Performance Benchmark White Paper
Inbound New Email Performance
0
5
10
15
20
25
30
No Attachment 120 KB
Mes
sage
s per
sec
New Email Performance - Attachment Size
Sync Async
0
5
10
15
20
25
30
No Attachment 120 KB
Mes
sage
s per
sec
New Email Performance - Impact of Archival
Without Archival With Archival
EngageOne Deliver Performance White Paper Page 26 of 40
Performance Benchmark White Paper
Mobile Push Notification Testing This section presents performance testing results for Mobile Push Notifications. Tests were run for FCM, APNS and AWS gateways. JMeter was used for hitting web services.
Test Results
Gateway Type
Rest Service Max Thread Pool Size
# Users/Threads in JMeter # Hits/User Average TPS (Message
delivered per sec)
APNS 20 10 5000 50
APNS 30 10 5000 55
AWS APNS 20 10 5000 11
AWS APNS 30 10 5000 21
FCM 20 10 5000 40
FCM 30 10 5000 50
AWS FCM 20 10 5000 10
AWS FCM 30 10 5000 20
EngageOne Deliver Performance White Paper Page 27 of 40
Performance Benchmark White Paper
Mobile Push Notification Performance
0
10
20
30
40
50
60
APNS AWS APNS FCM AWS SNS
TPS
Mobile Push Notification Performance - Mobile Push Channels
Thread Pool Size = 30No. of Users = 5No. of Hits per User = 5000
0
10
20
30
40
50
60
APNS AWS APNS FCM AWS SNS
TPS
Mobile Push Notification Performance - Thread Pool Size
Thread Pool Size 20 Thread Pool Size 30
No. of Users = 5No. of Hits per User = 5000
EngageOne Deliver Performance White Paper Page 28 of 40
Performance Benchmark White Paper
System Load Results This section presents the system load in terms of memory and CPU utilization.
The following parameters were observed while processing a number of scenarios. Statistics were captured through the Windows Task Manager utility.
• CPU Utilization • Memory Consumption • Network IO
Screen shots were taken during the main processing time period of each particular scenario. The period selected represents the most frequent utilization on average for the relevant scenario. Note that results are captured for the EngageOne Deliver Application Server (Tomcat), Mail / SMS server and Database machine.
EngageOne Deliver Performance White Paper Page 29 of 40
Performance Benchmark White Paper
Scenario 1: Email, Async mode, 5 files, 5 threads Tomcat Server machine
EngageOne Deliver Performance White Paper Page 30 of 40
Performance Benchmark White Paper
hMailServer machine
Database machine
EngageOne Deliver Performance White Paper Page 31 of 40
Performance Benchmark White Paper
Scenario 2: Email, Async mode, 5 files, 5 threads, 120 KB attachment Tomcat Server machine
hMailServer machine
EngageOne Deliver Performance White Paper Page 32 of 40
Performance Benchmark White Paper
Scenario 3: Email, Async mode, 5 files, 5 threads, 120 KB attachment, Digital Signature Tomcat Server machine
hMailServer machine
EngageOne Deliver Performance White Paper Page 33 of 40
Performance Benchmark White Paper
Scenario 4: Email, Async mode, 5 files, 5 threads, 1 MB attachment Tomcat Server machine
hMailServer machine
EngageOne Deliver Performance White Paper Page 34 of 40
Performance Benchmark White Paper
Scenario 5: SMS, 10 files, 5 threads Tomcat Server machine
EngageOne Deliver Performance White Paper Page 35 of 40
Performance Benchmark White Paper
SMPP Simulator machine
Database machine
EngageOne Deliver Performance White Paper Page 36 of 40
Performance Benchmark White Paper
Scenario 6: SMS, 10 files, 5 threads, Archival ON Tomcat Server machine
SMPP Simulator machine
EngageOne Deliver Performance White Paper Page 37 of 40
Performance Benchmark White Paper
EngageOne Deliver Performance Recommendations Note: You can also review the recommendations in the EODeliver Installation Guide.
The primary factors for achieving maximum performance include the following:
1. Disk Drive IOPS: Persistent storage performance is vital for achieving optimum results. High Performance RAID storage with Serial Attached SCSI (SAS) drives, such as 15K RPM Hard Drives or High Performance SLC SSD is highly recommended.
2. CPU (quantity and speed): Due to the multi-threaded architecture of Deliver application, 8 processors (2 Quad cores) or more are recommended.
3. RAM (quantity and speed): To take advantage of file system caching and to provide ample memory for JVM to allow internal caching, 32 GB or more of high-speed RAM is recommended.
4. Multiple disks: If separate and independent high-performance drives are available, configure each of the following directories on a separate drive but on the same server. If only one high performance drive is available, keep all directories on that drive.
• Directories containing the DIJ, HTML, text, images, error, and vendor files
• Archive Folder (if archiving enabled)
5. Latency to the Email server and SMS gateways should be kept to a minimum; 10 ms maximum, zero latency when possible.
6. Bandwidth to the Email server and SMS gateways should be sufficient to handle the expected amount of traffic.
7. It is recommended to use a connection with a speed of at least double the expected traffic size since any binary data (such as PDF attachments) must be UUEncoded; this increases the size of the email. For example, if your email body is 15KB and your attachment is 120KB, your total email size will be 135KB. If you want to send 10 per second you would need at least 2.7MB/sec Internet connectivity (2 * 10 * 135KB/sec. = 2.7MB/sec). Note if emails and SMS will be sent and received simultaneously ensure your bandwidth requirements include both.
8. Firewalls can reduce performance and should be avoided wherever possible.
9. Ensure that the Deliver database purge scripts are set to run on a regular basis.
EngageOne Deliver Performance White Paper Page 38 of 40
Performance Benchmark White Paper
10. Regularly perform server Health Check and Database maintenance activities:
• Update Statistics, Purge SQL Logs and tempDB
• Application logs - Review logging levels
• Disk Defragmentation
EngageOne Deliver Performance White Paper Page 39 of 40
Performance Benchmark White Paper
Configuration considerations for optimal performance In order to achieve the above stated performance numbers, the following settings have been configured in accordance with the test environment used for this performance benchmark test. These values may not work for all environments. However, whenever a new environment is tuned, you should typically targeted these settings first:
• The number of threads configured in the outboundProcessor.properties file should be equal to the number of DIJs (Journal files) to be executed. For more information related to this property, refer EODeliver Reference Guide.
• While installing Deliver, you can set the Root Folder (vendor folder) locally or remotely depending on which provides a desirable latency time of 1 ms or less. For more information, please refer EODeliver Installation Guide.
DIJ based email outbound processing can be run in legacy sync mode and more advanced async mode. Async mode is the default and preferred method.
Running email outbound processing in Async mode With the release of 1.5.1.0, a new Async mode was introduced for email outbound processing. This effectively separated the email composition process from the email sending process. This helps in configuring sending threads (or effective email gateway connection) separately, irrespective of the processing threads setting. This may provide a considerable performance improvements where higher email gateway capacity is available. For example, the availability of high speed bandwidth and email gateway response time. Improved performance may be observed where more than one email gateway is available to EngageOne Deliver.
Note that the Async mode setting is valid for DIJ file based email processing only. REST APIs always work in async mode.
Async mode can be enabled through a property in asyncProcesses.properties file.
# define process type values sync and async outbound.process.type=async
Detailed configurations for sending thread and pool size can also be applied in this file. Refer to the EODeliver Reference guide for details.
EngageOne Deliver Performance White Paper Page 40 of 40
Performance Benchmark White Paper
Settings for parallel profile threads and gateway connections The following table presents a sample method for calculating the required Email/SMS connection pool settings based on the outbound profile and number of profile threads. Note that Email and SMS connection pool settings need to be adjusted based on the load and available gateway capacity.
Table 1: Parallel Thread Settings
No of active e-mail profiles [X]
E-mail parallel threads [A]
Pool requirement for e-mail [X*A]
No. of active SMS profiles [Y]
SMS parallel threads [B]
Pool requirements for SMS [Y*B]
Total pool size [(X*A)+(Y*B)]
No. of parallel connections allowed at EngageOne Deliver email gateway [1.2*(X*A)]
10 1 10 1 1 1 11 12
10 2 20 1 2 2 22 24
10 3 30 2 1 2 32 36
20 1 20 2 2 4 24 24
20 2 40 3 1 3 43 48
20 3 60 3 2 6 66 72
50 1 50 3 3 9 59 60
50 2 100 5 1 5 105 120
50 3 150 5 2 10 160 180
50 3 100 5 3 15 115 120