marcelo r.n. mendes. what is fincos? a java-based set of tools for data generation, load submission,...

FINCoS Framework Tutorial

FINCoS Framework TutorialMarcelo R.N. Mendes

1What is FINCoS?A Java-based set of tools for data generation, load submission, and performance measurement of event processing systems;Main Characteristics:Flexible;Independent of particular workloads;Not bound to any specific CEP product;Scalable.2CISUC, University of Coimbra - mnunes@dei.uc.ptJuly, 20122

FINCoS: Architecture3CISUC, University of Coimbra - mnunes@dei.uc.ptJuly, 20123FINCoS: ComponentsDriverSimulates external sources of events;Submits load to the SUT;Includes data generation mechanism;SinkReceives query results from CEP engine;Stores output events in log files for subsequent validation.

4CISUC, University of Coimbra - mnunes@dei.uc.ptJuly, 2012

4FINCoS: ComponentsControllerMain Interface with the user. Allows to:Create reusable test setups;Control Drivers and Sinks (e.g., load, start, stop);Alter workload parameters on-the-fly (e.g., event submission rates, switch phases)

5CISUC, University of Coimbra - mnunes@dei.uc.ptJuly, 2012

5FINCoS: ComponentsPerformance MonitorCollect Metrics of Performance RunsTwo Modes:Online (integrated into FINCoS Controller);Offline (log files).

6CISUC, University of Coimbra - mnunes@dei.uc.ptJuly, 2012

6Lesson 1: Creating a Test SetupConfigure a DriverDecomposable WorkloadSynthetic vs. Thirdy-Party DatasetData Generation OptionsEvent Arrival ProcessConfigure a SinkJuly, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt77July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt8

8July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt9

9July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt10

unique identifiernumber of threads used for load generationsave generated data to disk?to where Driver send the eventsmachine that hosts the Driver10July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt11

Workload can be split in independent phases 11July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt12

12July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt13

Chooses the type of workload for this phaseSets events arrival processEvent MixEvents are generated in a ordered wayWhen data generation happensTo ensure repeatability13July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt14

Configure event types14July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt15

15July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt16

16July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt17

Configure Event Type Schema17July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt18

18July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt19

19July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt20

Column Name

20July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt21

Column Type

21July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt22

Data Generation Options

22July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt23

Add Values23July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt24

24July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt25

25July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt26

26July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt27

27July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt28

28July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt29

29July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt30

30July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt31

31July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt32

32July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt33

33July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt34

34July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt35

35July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt36

36July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt37

unique identifierList of event types this Sink subscribes tofrom where Sink receive eventsmachine that hosts the Sinksave generated data to disk?37July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt38

38July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt39

39July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt40

40July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt41

Done!41Lesson 2: Configuring a new ConnectionConfigure a reusable connection with a CEP engine or JMS ProviderJuly, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt4242July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt43

43July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt44

44July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt45

45July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt46

unique identifiername of the CEP engine (out-of-the-box support for Esper)Properties used to connect with the CEP enginetype of the connection46July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt47

* supporting new CEP engines requires coding47July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt48

Done!48Lesson 3: Running TestsLoad Drivers and SinksStart Load SubmissionControl Load Submission

NOTE: At this point the following steps must have been completed:Creation of a test setup;Initialization of the FINCoS Daemon Service at the machines where Drivers and Sinks are intended to run;July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt4949July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt50

Load components50July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt51

51July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt52

Start Load Submission52July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt53

Sent/Received counts53July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt54

54July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt55

55July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt56

56Lesson 4: Collecting Performance MetricsOnline Performance MeasurementDuring tests

Offline Performance MeasurementAfter test completion, use FINCoS Performance Monitor to process log files

July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt5757July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt58

58July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt59

July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt60

60Lesson 4: Collecting Performance Metrics (...cont)Online Performance MeasurementDuring tests

Offline Performance MeasurementAfter test completion, use FINCoS Performance Monitor to process log files

July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt6161July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt62

July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt63

July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt64

Configure M.I. here:Optionally, save collected metrics to a fileJuly, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt65

July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt66

Select the stream66July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt67

Select the metric67July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt68

Select exhibition scale68July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt69

Add Metrics to the Chart69July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt70

Change Visualization Options70Lesson 5: Setting Test OptionsResponse Time MeasurementJuly, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt7171July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt72

72July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt73

73July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt74

74Response Time Measurement ModeJuly, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt75DRIVERCEP ENGINESINKADAPTER234sendconvertinputconvertoutputprocessinputforwardgenerate resultforwardsend175Response Time Measurement ModeJuly, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt76DRIVERCEP ENGINESINKADAPTERsendconvertinputconvertoutputprocessinputforwardgenerate resultforwardsendt = t4 t1End-to-end:From Driver up to Sink 234176Response Time Measurement ModeJuly, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt77DRIVERCEP ENGINESINKADAPTERsendconvertinputconvertoutputprocessinputforwardgenerate resultforwardsendt = t3 t2Process Time:From immediately before the input event is forwarded to CEP engine to immediately after the result is received.234177Download and FeedbackFINCoS is available at:http://code.google.com/p/fincos/

Any suggestions or bug reports:mnunes@dei.uc.pt

July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt78