Using transactional BPM for serviceorchestration on NoSQL

Joram Barrez – Principal Engineer

Paul Holmes-Higgin - Chief Product Officer

What is it? How is this useful to me? Architecture and some code samples.

Flowable

Our journey: implementation details – challenges & pitfalls - benefits?

MongoDB

Scope & Goal (we’ve got lots to cover)

2

MicroservicesWhat does this mean for Flowable?

Manage complex business activities that require

a mix of human and electronic actions, such as an incoming application, a submitted claim, a

complaint, or incident management

Case management

Coordinate or orchestrate the behaviour of

people, systems, information, and things to produce business outcomes. Processes are

typically structured and repeatable

Business process management

Case and Business Process ManagementThe heart of Digitalization

3

BPMNCMMN

DMN

4

Open Standards

Flowable: the Project

• Open Source• Fork of Activiti by the Activiti developers• Active forums and contributions

• BPMN 2.0, CMMN 1.1 and DMN 1.1 engines• High performance, high scale, years of real world use

• Last release 6.4.0• https://github.com/flowable

• https://flowable.org/

5

Flowable Engines in a nutshell

Models

Deployed- Standalone (e.g REST)- Embedded in any Java env(with mimimal resources needed)

Engines APIs

Process (bpmn)Case (cmmn)Rules (dmn)

- Web apps- Mobile- Machine to Machine

- Open standards- Pluggability & extensibility

What does it give me as a developer?

- Yuck, visual programming!

- The model is “metadata”. It’s the glue between business and dev. It takes away (a big part of) the dreaded requirements document(s).

- Flowable makes working with these models easy and developer-friendly

- Flowable allows you to write unit tests, deploy and work with these models and its instances in a dev-natural way

DIY quick demo

8

Code needed to create a REST endpoint that starts a process instance

Role in a microservices world

• As an orchestrator/conductor: a central component managing a business goal that stretches across service boundaries, keeping services simple and focused.

9

Flowable Engine

Service A

Service B

Service Z

…

Role in a microservices world

• As an orchestrator/conductor: a central component managing a business goal that stretches across service boundaries, keeping services simple and focused.

10

Flowable Engine

Service A

Service B

Service Z

…

events

Role in a microservices world

• As an aggregator

11

New service

Service A

Service B

Service Z…

Role in a microservices world

• As exposing one service, where the function or domain just happens to be implemented by a process engine

12

Evolution of our own architecture

DMN

Form

JobTaskVariable Content

Content

BPMN

FormIDM

CMMN

DMN

BPMN

DMN Form

FormIDMBPMN

REST API

Engine

Service

Spring Boot

OSGI

LDAP

JMX

CDI

Identity Link Camel

Integration

Persistence

Task Job IDM VariableIDM

…

CMMN

CMMN

MongoDB• Flowable relies on the transactional semantics of a

relational db

• “Atomically” moving from one stable state to another

• This doesn’t free you from forgetting about service failures,

but understanding the transactional model of Flowable sure

makes it easier to write resilient processes

• MongoDB 4.0 added support for transactions (June)

14

MongoDB• Open-source NoSql document store

• Short history

• Started in 2007 by 10gen as component of their PaaS

• Was known in the early days (2.2 versions and before) as the dev/null db

• Acquired WiredTiger end of 2014

• WiredTiger default storage engine in 3.2

• WiredTiger enables transactional semantics (ACID) on multi-document operations in 4.0 (*)

15* “Path to transactions” series on https://www.youtube.com/user/MongoDB/videos

Flowable – MongoDB• All code: https://github.com/flowable/flowable-mongodb

• 6.0.4.alpha1 release last week

16

Service call

Command Interceptor/ Commands

Agenda / operations

EntityManagers

DataManagers

Engine core logic

Low-level data access

High-level data functions

Implementation

• MongoDB’s transactions follow a familiar programming model

• Concept of clientSession

• Matches Flowable’s low-level session concept nicely

17

Demo

18

Relational vs MongoDB implementation

Implementation• Replace all Datamanager interface implementations with

a MongoDB counterpart

• -> alpha releases

• Using the existing test suite to validate the implementation

• Completed -> beta / stable release

• (Almost) 1-1 translation of the relational data structure

• Optimizations along the way

• MongoDb-specific structure optimization surely will follow19

Challenges

• com.mongodb.MongoCommandException: Command failed with error 112 (WriteConflict): 'WriteConflict' on server exethanter.local:27017. The full response is { "errorLabels" : ["TransientTransactionError"], "operationTime" : { "$timestamp" : { "t" : 1537701066, "i" : 3 } }, "ok" : 0.0, "errmsg" : "WriteConflict", "code" : 112, "codeName" : "WriteConflict", "$clusterTime" : { "clusterTime" : { "$timestamp" : { "t" : 1537701066, "i" : 3 } }, "signature" : { "hash" : { "$binary" : "AAAAAAAAAAAAAAAAAAAAAAAAAAA=", "$type" : "00" }, "keyId" : { "$numberLong" : "0" } } } }

20

Challenges

• Taking joins for granted

• Denormalization needed

• Way more work as a developer to guarantee data consistency

• E.g simple example: see ‘latest’ of Process definition

• Exchange writes/updates for faster reads

21

Luckily• Over the past years

• We’ve made Flowable a lot faster by keeping in mind that one exchange over a network is extremely expensive

• Denormalization, prefetching, entity counts

22

Performance

• Is the performance acceptable?

• Benchmark on AWS

• Setup (see GitHub repo)

23

Process Service

Postgres

MongoDB

m5d.2xlarge (8 cores/32Gb RAM), 100GB SSD

m5d.2xlarge (8 cores/32Gb RAM), 100GB SSD

t3.2xlarge (8 cores/32Gb RAM), 100GB SSD

max_connections = 100shared_buffers = 8GBeffective_cache_size = 24GBmaintenance_work_mem = 2GBcheckpoint_completion_target = 0.7wal_buffers = 16MBdefault_statistics_target = 100random_page_cost = 1.1effective_io_concurrency = 200work_mem = 20971kBmin_wal_size = 1GBmax_wal_size = 2GBmax_worker_processes = 8max_parallel_workers_per_gather = 4max_parallel_workers = 8listen_addresses = '*'

Process Service• Bring process into a stable state

• One transaction

• Fixed threadpool of 8 threads

24

- 6 executions- 2 user tasks- 1 hist. proc inst- 11 hist. activities- 2 hist. user tasks- 31 variables- 31 hist. variables- 1 timer job

Results

• Reverse of what we expected J

25

Results

• Reverse of what we expected J

• Although the graphs seem to indicate a relative large difference, we’re talking about sub-ms differences!

• Relational db’s have not been idling

• See our recent performance benchmarks

• https://blog.flowable.org/2018/03/05/flowable-6-3-0-performance-benchmark/

• https://blog.flowable.org/2018/03/13/async-history-performance-benchmark/

26

Conclusion

• The transactional support in MongoDB is impressive

• Data consistency perspective

• Performance perspective

• Using Flowable on MongoDB is a valid alternative

27

Current limitations

• Read/Write to primary only

• Adding replica nodes seemed to have a negative effect

• Even though read/write to primary (current MongoDB transactions limitation)

• MongoDB transactions are still under development

28https://www.youtube.com/watch?v=dQh03YLkmyg

Future work

• MongoDB is designed for horizontal scale

• (Yes, (for example) postgres has partitioning, but …)

• Sharded clusters + Flowable à interesting use cases

• Shard by tenant

• Shard on process definition key

• BigData use cases … like ML!

29

Machine Learning

• MongoDB being “BigData” (e.g better suited for streaming, reactive, etc.) opens up use cases for ML• Demo• Run processes a lot from start to end• Feed historical data into ML• See if human work is repetitive and suggest optimizations

30

Machine Learning

1. Look for Human Decision patterns

31

Machine Learning

1. Look for Human Decision patterns2. Gather possible data inputs and backtrack

Architecture

• vs last year

33

Architecture

34

Process Service

UI

Stream as RDD

Decision Analysis Service

suggestions

Spark (cluster) + MLlib

Process ServiceProcess

Service

Decision Analysis Service

Decision Analysis Service

DemoProcesses with Machine Learning

35

Source Codehttps://github.com/flowable/flowable-

examples/tree/master/demo/demo-jax-2018

36

Thank you!