tibco loglogic unity developer's guide · 2015. 12. 3. · tibco loglogic® unity...

TIBCO LogLogic® Unity Developer's GuideSoftware Release 2.1December 2015

Two-Second Advantage®

Important Information

SOME TIBCO SOFTWARE EMBEDS OR BUNDLES OTHER TIBCO SOFTWARE. USE OF SUCHEMBEDDED OR BUNDLED TIBCO SOFTWARE IS SOLELY TO ENABLE THE FUNCTIONALITY(OR PROVIDE LIMITED ADD-ON FUNCTIONALITY) OF THE LICENSED TIBCO SOFTWARE. THEEMBEDDED OR BUNDLED SOFTWARE IS NOT LICENSED TO BE USED OR ACCESSED BY ANYOTHER TIBCO SOFTWARE OR FOR ANY OTHER PURPOSE.

USE OF TIBCO SOFTWARE AND THIS DOCUMENT IS SUBJECT TO THE TERMS ANDCONDITIONS OF A LICENSE AGREEMENT FOUND IN EITHER A SEPARATELY EXECUTEDSOFTWARE LICENSE AGREEMENT, OR, IF THERE IS NO SUCH SEPARATE AGREEMENT, THECLICKWRAP END USER LICENSE AGREEMENT WHICH IS DISPLAYED DURING DOWNLOADOR INSTALLATION OF THE SOFTWARE (AND WHICH IS DUPLICATED IN THE LICENSE FILE)OR IF THERE IS NO SUCH SOFTWARE LICENSE AGREEMENT OR CLICKWRAP END USERLICENSE AGREEMENT, THE LICENSE(S) LOCATED IN THE “LICENSE” FILE(S) OF THESOFTWARE. USE OF THIS DOCUMENT IS SUBJECT TO THOSE TERMS AND CONDITIONS, ANDYOUR USE HEREOF SHALL CONSTITUTE ACCEPTANCE OF AND AN AGREEMENT TO BEBOUND BY THE SAME.

This document contains confidential information that is subject to U.S. and international copyright lawsand treaties. No part of this document may be reproduced in any form without the writtenauthorization of TIBCO Software Inc.

TIBCO, LogLogic, and Two-Second Advantage are either registered trademarks or trademarks ofTIBCO Software Inc. in the United States and/or other countries.

Enterprise Java Beans (EJB), Java Platform Enterprise Edition (Java EE), Java 2 Platform EnterpriseEdition (J2EE), and all Java-based trademarks and logos are trademarks or registered trademarks ofOracle Corporation in the U.S. and other countries.

All other product and company names and marks mentioned in this document are the property of theirrespective owners and are mentioned for identification purposes only.

THIS SOFTWARE MAY BE AVAILABLE ON MULTIPLE OPERATING SYSTEMS. HOWEVER, NOTALL OPERATING SYSTEM PLATFORMS FOR A SPECIFIC SOFTWARE VERSION ARE RELEASEDAT THE SAME TIME. SEE THE README FILE FOR THE AVAILABILITY OF THIS SOFTWAREVERSION ON A SPECIFIC OPERATING SYSTEM PLATFORM.

THIS DOCUMENT IS PROVIDED “AS IS” WITHOUT WARRANTY OF ANY KIND, EITHEREXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OFMERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT.

THIS DOCUMENT COULD INCLUDE TECHNICAL INACCURACIES OR TYPOGRAPHICALERRORS. CHANGES ARE PERIODICALLY ADDED TO THE INFORMATION HEREIN; THESECHANGES WILL BE INCORPORATED IN NEW EDITIONS OF THIS DOCUMENT. TIBCOSOFTWARE INC. MAY MAKE IMPROVEMENTS AND/OR CHANGES IN THE PRODUCT(S)AND/OR THE PROGRAM(S) DESCRIBED IN THIS DOCUMENT AT ANY TIME.

THE CONTENTS OF THIS DOCUMENT MAY BE MODIFIED AND/OR QUALIFIED, DIRECTLY ORINDIRECTLY, BY OTHER DOCUMENTATION WHICH ACCOMPANIES THIS SOFTWARE,INCLUDING BUT NOT LIMITED TO ANY RELEASE NOTES AND "READ ME" FILES.

Copyright © 2014-2015 TIBCO Software Inc. ALL RIGHTS RESERVED.

TIBCO Software Inc. Confidential Information

2

TIBCO LogLogic® Unity Developer's Guide

Contents

TIBCO Documentation and Support Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4

Using the REST API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Constructing REST Requests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

REST API Endpoint ( baseurl ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Response Status Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Query Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7

LogLogic Unity Search Queries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Creating a Cached Query and Retrieving Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Refining Search Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

Creating a Forward-only Query and Retrieving Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Tail Queries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Correlation Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19

Event Correlation Language (ECL) Parameter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

Correlation Workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Creating, Updating, and Deleting Replay Instance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

Creating or Updating Real-time Instance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

Retrieving and Acknowledging Alerts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

3


TIBCO Documentation and Support Services

Documentation for this and other TIBCO products is available on the TIBCO Documentation site:

https://docs.tibco.com

Documentation on the TIBCO Documentation site is updated more frequently than any documentationthat might be included with the product. To ensure that you are accessing the latest available helptopics, please visit https://docs.tibco.com.

Product-Specific Documentation

Documentation for TIBCO products is not bundled with the software. Instead, it is available on theTIBCO Documentation site. To directly access documentation for this product, double-click thefollowing file:

TIBCO_HOME/release_notes/TIB_logu_version_docinfo.html

where TIBCO_HOME is the top-level directory in which TIBCO products are installed. On Windows,the default TIBCO_HOME is C:\tibco. On UNIX systems, the default TIBCO_HOME is /opt/tibco.The following documents for this product can be found in the TIBCO Documentation site:

● TIBCO LogLogic® Unity Installation and Configuration Guide

● TIBCO LogLogic® Unity User's Guide

● TIBCO LogLogic® Unity Developer's Guide

● TIBCO LogLogic® Unity Tutorials

How to Contact TIBCO Support

For comments or problems with this manual or the software it addresses, contact TIBCO Support:

● For an overview of TIBCO Support, and information about getting started with TIBCO Support,visit this site:

http://www.tibco.com/services/support

● If you already have a valid maintenance or support contract, visit this site:

https://support.tibco.com

Entry to this site requires a user name and password. If you do not have a user name, you canrequest one.

How to Join TIBCOmmunity

TIBCOmmunity is an online destination for TIBCO customers, partners, and resident experts. It is aplace to share and access the collective experience of the TIBCO community. TIBCOmmunity offersforums, blogs, and access to a variety of resources. To register, go to the following web address:

https://www.tibcommunity.com

4




http://www.tibco.com/services/support

https://support.tibco.com

https://www.tibcommunity.com

Using the REST API

You can use the Representational State Transfer (REST) API to develop a custom client application.

LogLogic® Unity provides REST APIs that a client application can use to invoke services using simpleHTTP methods. A catalogue of available REST resources and requests, organized by functions, isprovided.

LogLogic Unity API online documentation can be accessed using the following URLs:

● Query Node - http://<hostname>:9681/docs

● Correlation Node - http://<hostname>:9682/docs

This assumes that the default ports are used for these nodes. If you change the default ports, update theURLs accordingly.

Constructing REST RequestsREST API requests must be submitted in a specific format.

The format of a LogLogic Unity REST API request is:

<METHOD> <baseurl>/<basePath>?<query_parameters>

where:

● <METHOD> is the HTTP method to be used on the resource (GET, POST, PUT, or DELETE)

● <baseurl> is the REST API Endpoint ( baseurl ).

● <basePath> is the part of the path that identifies the required LogLogic Unity resource. It consistsof:

— a fixed part - for example, /api/v1/query when starting a query.

— followed by (if required) path parameters - for example, id when starting a query

● <query_parameters> identifies any parameters to be passed as part of the request. Unlessotherwise specified for a specific parameter, all query parameters are always optional. Multiplequery parameters should be separated by ampersand (&) characters.

For example, the GET query request shows how to start a business service called GET query:GET http://localhost:9681/api/v1/query/{id}

where:

● http://localhost:9681/ is the LogLogic Unity REST API endpoint.

● api/v1 is the fixed part of the <basePath>.

● query is the path parameter in the <basePath>.

● {id} is the parameter of the path in the <basePath>.

REST API Endpoint ( baseurl )A specific endpoint must be used when submitting REST API requests.

The endpoint on which all LogLogic Unity REST resources are exposed is:

protocol://host:port/api/v1

where:

5


● protocol is the protocol used to communicate with the LogLogic Unity runtime HTTP.

● host is the network name or IP address of the runtime.

● port is the port used by the LogLogic Unity runtime for incoming HTTP connections (exposed bythe HTTP connector shared resource). This is port 968x.

● api is the context root used by the LogLogic Unity REST API resource hierarchy.

● v1 is the API version number.

The LogLogic Unity REST API endpoint is referenced throughout the rest of this document as<baseurl>. If HTTPS is desired then an HTTPS Proxy must be used, for example, NGINX.

Response Status CodesAll REST API requests return a response status code.

The main HTTP status codes that may be returned by LogLogic Unity are:

HTTP status code Description

200 Request completed successfully.

204 Request completed successfully but no content available to return.

400 Bad request/ Invalid query.

401 Authentication failure, invalid access credentials.

403 Insufficient permission.

404 <Component> id not found.

406 Not acceptable.

500 Unspecified internal server error.

6


Query Node

These methods are to create and interact the results of the query.

All available resources for the Query Node are described below.

Resource Description

Query

POST query Create a query on Query node and returns a query ID forresults and the query result schema. The query ID willremain in the system until it is deleted.

DELETE query/{id} Delete query results of the specified query ID.

POST query/{id}/results Create a result set. It is equivalent to create sub-tabs of theresult.

The extended set of EQL operations are supported by theResultSet EQL syntax. For details, see ResultSet EQLSyntax.

POST query/{id}/result/{resultSetId} Modify the conditions on the sub-tabs of the result set. Youcan add/remove conditions of the results tab. The result setID can change due to this method.

The extended set of EQL operations are supported by theResultSet EQL syntax. For details, see ResultSet EQLSyntax.

GET query/{id}/result/{resultSetId} Get results for the query and result set.

GET query/{id}/forward Get results for the forward-only queries.

DELETE query/{id}/result/{resultSetId}

Delete the result set.

Query cannot be deleted.

GET query/{id}/result/{resultSetId}/columns

Get columns for the result set.

GET queries Get a list of active results. The submitted queries are activeuntil they are deleted using the DELETE request.

Domains

GET config/domains Get a list of domains for the user's tenant.

PUT config/domains/{domain} Add or update domain for the user's tenant.

DELET config/domains/{domain} Delete a domain for the user's tenant.

GET diskUse/domains/{domain} Get the disk used by domain for the user's tenant.

7


For more information about EQL syntax, refer to the Search Syntax Reference section in the TIBCOLogLogic® Unity User's Guide.

For detailed information for each method and sample output, refer to the online LogLogic Unity APIdocumentation using the following URL:

http://<hostname>:9681/docs

This assumes that the default port is used for this node. If you change the default port, update the URLaccordingly.

LogLogic Unity Search QueriesQuerying LogLogic Unity using the REST API is composed of two steps: first, create a query andsecond, retrieve results for the search query.

Creating a Query

This is a single synchronous call where the query is specified and it returns results immediately. Theresult can be:

● Fails (if the query is invalid, for example, syntax errors), in which case the error details will bereturned.

● Succeeds, in which case information about the query is returned, including the ID number of thequery and the schema of the results (that means which columns will be returned).

When the query creation succeeds, the call returns immediately, but in the background LogLogic Unityis already in the process of generating results.

Retrieving Results

The results can be composed of multiple rows, it is not practical to retrieve them in one single call. Youhave to make multiple calls to collect all of them.

Also, the query running in the background can take more or less time depending on how much data isthere and the complexity of the query. You should wait for the data to be available, so ideally this call isdone in a loop, invoking it repeatedly until you have retrieved all the results.

Querying Styles

The following two types are supported:

● Cached query: in this query all results for your query are cached in query node, no matter how big;then you can go back and forth over them, and further refine your search over those results byadding filters or aggregations, the same way it is done through the LogLogic Unity Web UI. Thequery remains in the query node until it is explicitly deleted.

● Forward-only query: in this query results are not cached, but rather directly handed through theREST API, one row after the other until you got all of them and then the query is finished. Youcannot go back to results you already received, and you cannot refine the search, but theperformance is better. The query is deleted automatically when you have collected all the results,but you can also delete it explicitly before that happens. Also, these queries are automaticallydeleted after some time of inactivity (by default, it is set to 20 seconds but it can be configured usingthe key services.rest.options.forwardQuery.maxInactiveLife).

You should run Forward-only query if you just want to get the results of your query and you must runCached query if you want to go back and forth (for example, you are offering a scrollable view and youdo not want to or cannot cache the results in the client), or if you want to further refine the search.

8


Creating a Cached Query and Retrieving ResultsUsing the REST API, create a query on the Query node and it returns a query ID (used for retrievingresults) and the result schema. The query ID will remain in the system until it is deleted. You canretrieve results for a specific query ID later.

For parameter details, refer to the online LogLogic Unity API documentation.

The following example is to create a query using the POST query request.

RequestURL

POST <baseurl>/api/v1/query

RequestBody

{ "query": "use sample |sys_eventTime in -1y"}

ResponseBody

{ "id": "4", "resultSetId": "0", "columnDescriptors": [ { "name": "ll_eventAction", "type": "STRING", "clickable": false, "hasStats": true }, { "name": "ll_eventActionID", "type": "LONG", "clickable": false, "hasStats": true }, { .... } ], "type": "Query"}

When creating a query, the filter on sys_eventTime is mandatory. By default, the query will cache andreturn a maximum of 10,000 (totalCount) results, but that can be changed with the LIMIT statement.

Once the query is created, the results must be retrieved by repeatedly invoking the REST API. Thefollowing example is to retrieve results for the query ID 4 using the GET request. Note the parametersfrom and size as part of the URL, that means that we are requesting to retrieve 100 rows, starting fromthe row 0.

RequestURL

GET <baseurl>/api/v1/query/4/result/0?from=0&size=100

9


ResponseBody

{ "id": "4", "resultSetId": null, "from": 0, "count": 100, "totalCount": 9932, "eventsProcessed": 9932, "eventsTotalCount": 9932, "results": [ [ "Successful Network Logon", 2, 13, "172.172.172.172", ..., "<13>Feb 4 09:40:44 oktulsaew007.exttul.aepext.com MSWinEventLog\t1\tSecurity\t1545726\tTue Feb 02 12:44:35 2014\t540\tSecurity\tSiteSvrAdmin\tUser\tSuccess Audit\tOKTULSAEW007\tLogon/Logoff\t\tSuccessful Network Logon: User Name: SiteSvrAdmin Domain: OKTULSAEW007 Logon ID: (0x0,0x64E992A) Logon Type: 3 Logon Process: NtLmSsp Authentication Package: NTLM Workstation Name: OHAEPHQIW005 Logon GUID: - Caller User Name: - Caller Domain: - Caller Logon ID: - Caller Process ID: - Transited Services: - Source Network Address: 10.92.4.147 Source Port: 0 \t1544545", 611, 115813, 1422381393642, "", "", 65536, "172.172.172.172", "com.myworkstation", "/home/toor/GABuild/logu/1.0/tools/samples/sample/sample.log", 9800 ], [ .... ], "statsData": null, "chartData": null, "messages": [ { "text": "Query completed in 15 seconds and 817 milliseconds", "severity": "INFO" } ] }

Note that in the results, the totalCount is 9932. That means that the query is completed, and the totalnumber of rows for the results is 9932. However, in this request we requested for 100 events to bereturned, starting from the row 0. In successive calls, you can get the remaining rows in the results. Forexample, in the next call you can start with from=100, and size=100 to request the next one hundredresults.

If the totalCount shows -1, that means the query is still running, so you should keep query calling untilit is completed. You must leave some time between calls, otherwise the server will be very busyresponding to your requests and that can hurt the query performance. So it is a good practice that,between two calls, code "sleeps" for some milliseconds, for example, 200. Eventually, the totalCount willbe something different than -1, showing that the query is completed.

You can evaluate when the query can be completed by checking the eventTotalCount and eventsProcessedfields. The eventTotalCount field is the total number of events that need to be processed for this queryand the eventsProcessed field is the number of events processed until now. The eventsProcessed growsuntil it eventually reaches the eventTotalCount.

The following example shows the query is still running since the totaCount is -1. The eventsProcessedcount is lower than the eventTotalCount , which helps you calculate the progress of the query.

10


RequestURL


ResponseBody

{ "id": "167", "resultSetId": null, "from": 0, "count": 0, "totalCount": -1, "eventsProcessed": 9932, "eventsTotalCount": 218467, "results": [], "statsData": null, "chartData": null, "messages": null}

The following example shows results retrieved from row 100 to 250 with MESSAGES, STATS, andCHART. The Stats will return the statistics about the columns and Charts are visible on the web UI.

RequestURL


11


ResponseBody

{ "id": "4", "resultSetId": null, "from": 100, "count": 150, "totalCount": 9932, "eventsProcessed": 9932, "eventsTotalCount": 9932, "results": [ [ "Successful Network Logon", 2, 13, "172.172.172.172", "<13>Feb 4 09:40:45 oktulsaew007.exttul.aepext.com MSWinEventLog\t1\tSecurity\t1546153\tTue Feb 02 13:57:56 2014\t540\tSecurity\tSiteSvrAdmin\tUser\tSuccess Audit\tOKTULSAEW007\tLogon/Logoff\t\tSuccessful Network Logon: User Name: SiteSvrAdmin Domain: OKTULSAEW007 Logon ID: (0x0,0x68F5B3E) Logon Type: 3 Logon Process: NtLmSsp Authentication Package: NTLM Workstation Name: OHAEPHQIW005 Logon GUID: - Caller User Name: - Caller Domain: - Caller Logon ID: - Caller Process ID: - Transited Services: - Source Network Address: 10.92.4.147 Source Port: 0 \t1544972", ... ], [ ...... ], "statsData": [ 4, 1, 1, 1, ........ ], "chartData": [ [ 1391373875000, 42 ], [ 1391375674641, 46 ], [ .................. ], "messages": null }

Java Examplepackage com.tibco.sample;

import static com.jayway.restassured.RestAssured.given;

import java.util.ArrayList;import java.util.Iterator;import java.util.List;

import com.fasterxml.jackson.databind.JsonNode;import com.fasterxml.jackson.databind.ObjectMapper;import com.fasterxml.jackson.databind.node.JsonNodeFactory;import com.fasterxml.jackson.databind.node.ObjectNode;import com.jayway.restassured.response.Response;

public class CachedQuerySample {

public static void main( final String[] args ) throws Exception {

12


final String query = "use unity | sys_domain='internal' " + "| columns sys_body | sys_eventTime in -1h"; final ObjectMapper mapper = new ObjectMapper();

// create the query final ObjectNode jsonNode = JsonNodeFactory.instance.objectNode(); jsonNode.put( "query", query );

final Response createQueryResponse = given().auth().basic( "admin", "admin" ) .contentType("application/json;charset=utf-8") .body(jsonNode.toString()).when() .post("http://localhost:9681/api/v1/query");

final JsonNode root = mapper.readTree( createQueryResponse.asString() ); final String queryId = root.get( "id" ).asText();

// call until the query is completed int totalCount = -1; for ( int i = 0; i < 20; i++ ) { final Response response = given().auth().basic( "admin", "admin" ) .pathParameter("queryId", queryId).queryParameter("size", "1") .queryParameter("type", "MESSAGES").when() .get("http://localhost:9681/api/v1/query/{queryId}/result/0");

final JsonNode apiResult = mapper.readTree( response.asString() ); if ( apiResult.get( "totalCount" ).asInt() != -1 ) { totalCount = apiResult.get( "totalCount" ).asInt(); break; }

Thread.sleep( 200 ); }

// get the results in batches of 500, and print them for ( int i = 0; i < totalCount; i += 500 ) {

final Response response = given().auth().basic( "admin", "admin" ) .pathParameter( "queryId", queryId ) .queryParameter( "size", "500" ) .queryParameter( "from", i ) .queryParameter( "type", "MESSAGES" ) .when() .get( "http://localhost:9681/api/v1/query/{queryId}/result/0" );

int j = 0; final JsonNode apiResult = mapper.readTree( response.asString() ); final Iterator<JsonNode> rowIterator = apiResult.get( "results" ).elements(); while ( rowIterator.hasNext() ) { final List<String> row = new ArrayList<String>(); final Iterator<JsonNode> colIterator = rowIterator.next().elements(); while ( colIterator.hasNext() ) { row.add( colIterator.next().asText() ); } System.out.println( "Row " + (i+j) + ": " + row ); j++; } } }}

13


Refining Search ResultsTo perform filtering and group-by operations in the result set, an extended set of EQL operations havebeen defined, which can be used in conjunction with the standard EQL syntax.

This following syntax is only applicable when running these queries: POST query/{id}/results andPOST query/{id}/result/{resultSetId}. For detailed information about the EQL syntax, refer to the SearchSyntax Reference section in the TIBCO LogLogic® Unity User's Guide.

To perform time-range filters, the standard TIMESTAMP BETWEEN EQL syntax can be used.<timestampColumn> BETWEEN [startTimeInMillis] AND [endTimeInMillis]

For example,

sys_eventTime BETWEEN 123475666 AND 234334334

To perform column value include or exclude filtering, the standard EQL '=' or '!=' syntax can be used.col1 = <columnName>col2!= <columnValue>

Examples:

col1 = 'MyColumn'

col2!= 34

To perform message body include or exclude filtering, the EQL LIKE syntax can be used.<columnName> LIKE <'string'> / <columnName> NOT LIKE <'string'>

Examples:

sys_body LIKE 'str'

sys_body NOT LIKE 'str' (for excluding)

All of the filter expressions listed above can be combined using AND or OR.

For example:

(sys_body LIKE 'str') AND (col1 = 'MyColumn')

To perform group by, and aggregation , and drilldown of a group-by value, the following can be used:GROUP MESSAGES BY <columnName>GROUP MESSAGES BY <timestamp_column> HAVING TIMERANGE INTERVAL [SECOND|MINUTE|HOUR|DAY|WEEK|MONTH]GROUP MESSAGES BY <columnName> AGGREGATE USING SUM(columnname)GROUP MESSAGES BY <columnName> AGGREGATE USING MIN(columnname),MAX( columnname) ==> can specify more thanone aggregation column (one of MIN, MAX, AVG, or SUM)GROUP MESSAGES BY <columnName> | DRILLDOWN USING <drilldown value>

Examples:

GROUP MESSAGES BY ll_sourceUser

GROUP MESSAGES BY sys_eventTime HAVING TIMERANGE INTERVAL HOUR

GROUP MESSAGES BY (ll_sourceUser) AGGREGATE USING MIN(sys_eventTime),

MAX(sys_eventTime)

To perform sorting, the standard EQL sort syntax can be used:SORT BY <columnName> [ASC|DESC]

For example:

14


SORT BY sys_eventTime ASC

Creating a Forward-only Query and Retrieving ResultsIn regular queries, once the query is created, you can get any portion of the results at any time. In theForward-only queries, results are collected in batches, from the first to the last until there are no moreresults. It is not possible to scroll-back and get results that have been already collected.

Forward-only queries offer a better performance since results are not cached but they impose thefollowing restrictions:

● No scrolling back● No further filtering or grouping over the result set● No stats or charts are returned

The following example is to create a forward-only query using the POST query request with specifiedthe FORWARD_ONLY style.

RequestURL

POST <baseurl>/api/v1/query

RequestBody

{ "query": "use sample |sys_eventTime in -1y","style": "FORWARD_ONLY"}

ResponseBody

{ "id": "4", "resultSetId": "0", "columnDescriptors": [ { "name": "ll_eventAction", "type": "STRING", "clickable": false, "hasStats": true }, { "name": "ll_eventActionID", "type": "LONG", "clickable": false, "hasStats": true }, { .... } ], "type": "Query"}

To collect the results, you need repeatedly invoke the following:

RequestURL

GET <baseurl>/api/v1/query/5/forward?minEvents=1&maxEvents=100&timeout=100

15


ResponseBody

{"results": [["Successful Network Logon",2,13,"172.172.172.172",540,1300089,"134"...],...],"messages": [],"finished": false,"queryProgress": 100,"eventsToProcess": 4002}

This query has the following 4 parameters:

● minEvents means the minimum count of rows to collect; the server will hold the http request ifthere are not that many rows available, until they come, or the timeout passes. Default is zero,which means the server will not wait for rows to be available. It will provide whatever it has at themoment, even if it is zero rows.

● timeout is the number of milliseconds the server will hold the request while waiting to haveminEvents available. If this time passes, the server will return whatever data is available, but it willreturn earlier if it can get those rows before. Default is 0, meaning no wait at all.

● maxEvents is the maximum number of rows that the server will respond with. Even if the server hasmore rows available, it will never return more than this. Note that rows that are not returned in arequest due to maxEvents limit will be returned in subsequent requests. Rows are returned in order,from the first to the last. This parameter is useful to control how many rows you will receive at atime, so that it does not block the client system. The default value is 1000.

● showProgressdescribes whether to return progress information in the response. If the value is true,the two fields are returned: queryProgress shows the percentage of progress for this query andeventsToProcess shows how many events need to be processed to process this query. The defaultvalue is set to true.

The "results" array contains a batch of rows in the result; the "messages" array contains any messageslike errors that the server wants to send to the client, and the finished flag will be true when the queryis finished. You are supposed to keep on invoking this operations until the "finished" flag is true; therows that you get in each invocation are correlative to the previous ones.

For example, if you create a forward-only query that is supposed to retrieve 30 results. You invoke thisoperation repeatedly:

● 1st invocation: { "results": [], "messages": [], "finished": false }● 2nd invocation: { "results": [...<5 rows>...], "messages": [], "finished": false }● 2nd invocation: { "results": [...<13 rows>...], "messages": [], "finished": false }● 3nd invocation: { "results": [...<12 rows>...], "messages": [], "finished": true }

The number of rows in each invocation varies based on how many events are available at that time. Inthe first invocation, no results are returned because they are not yet available. In subsequentinvocations, batches of rows of different sizes are returned. You are supposed to keep them in order soyour total results are: <5 rows from 1st invocation><13 rows from 2nd invocation><12 rows from 3rdinvocation> which make the 30 rows expected.

In the last request the response contains the flag "finished": true, which means that there are no morerows available. The query is finished and you do not need to make this request any more. The several

16


batches of rows that you have received, together, are the result of your query. You only view that thequery is finished when the response returns "finished": true.

Java Examplepackage com.tibco.sample;

import static com.jayway.restassured.RestAssured.given;

import java.util.ArrayList;import java.util.Iterator;import java.util.List;

import com.fasterxml.jackson.databind.JsonNode;import com.fasterxml.jackson.databind.ObjectMapper;import com.fasterxml.jackson.databind.node.JsonNodeFactory;import com.fasterxml.jackson.databind.node.ObjectNode;import com.jayway.restassured.response.Response;

public class ForwardOnlyQuerySample {

public static void main( final String[] args ) throws Exception {

final String query = "use unity | sys_domain='internal' " + "| columns sys_body | sys_eventTime in -1h";

final ObjectMapper mapper = new ObjectMapper();

// create the query final ObjectNode jsonNode = JsonNodeFactory.instance.objectNode(); jsonNode.put( "query", query ); jsonNode.put( "style", "FORWARD_ONLY" );

final Response createQueryResponse = given().auth().basic( "admin", "admin" ) .contentType( "application/json;charset=utf-8" ) .body( jsonNode.toString() ).when() .post( "http://localhost:9681/api/v1/query" );

final JsonNode root = mapper.readTree( createQueryResponse.asString() ); final String queryId = root.get( "id" ).asText();

// get the results in batches (size decided by the server based // on how many are available), and print them int i = 0; while ( true ) {

final Response response = given().auth().basic( "admin", "admin" ) .pathParameter( "Id", queryId ).queryParameter( "minEvents", 1 ) .queryParameter( "maxEvents", 500 ) .queryParameter( "timeout", 5000 ) .when() .get( "http://localhost:9681/api/v1/query/{Id}/forward" );

final JsonNode apiResult = mapper.readTree( response.asString() );

final Iterator<JsonNode> rowIterator = apiResult.get("results").elements(); while ( rowIterator.hasNext() ) { final List<String> row = new ArrayList<String>(); final Iterator<JsonNode> colIterator = rowIterator.next().elements(); while ( colIterator.hasNext() ) { row.add( colIterator.next().asText() ); } System.out.println( "Row " + i + ": " + row ); i++; }

17


if ( apiResult.get( "finished" ).asBoolean() == true ) { break; } } }}

Tail QueriesTail queries are queries that run on real-time data, instead of on historical data as regular queries. Onceyou create a tail query, it will return new events coming into the system that matches the query criteria.The events will appear in the query results in real-time, as they are ingested into Unity. Because of thenature of these queries, they never finish: the user has to cancel/delete them manually.

Tail queries have the following restrictions:

● always sorted by time● group-by or aggregation functions are not supported

To create a tail query include the condition sys_eventTime > now, as shown in the following example:

select * from sample where sys_body LIKE '%failed%' and sys_eventTime > now

You can also use the tail statement (only available in EQL) as shown below:

use sample | sys_body LIKE '%failed%' | tail

This is controlled via EQL or SQL. There are no changes to the API usage, and both regular andforward-only queries can work with tail queries. ResultSet filtering is not available for tail queries.

18


Correlation Node

All available resources for the Correlation Node are described below.

Resource Description

Instances

GET activeInstances Get a list of active instance states that currently exist in thenode.

GET instances Get a list of instances that currently exist in the node.

POST instances Create a new instance and run it on the specified data.

GET instance/{id} Get a summary of an instance.

POST instance/{id} Update the state of an instance.

DELETE instance/{id} Delete an instance from the node.

GET instance/{id}/correlationevents Get correlation events for an instance. This supports onlyreplay instances.

GET instance/{id}/correlationevent/{eventId}/group/ {name}/eventKeys

Get a list of event references and sources. This supportsonly replay instances.

GET instance/{id}/result/{resultSetId}/columns

Get lists of correlation result set columns. This supportsonly for replay instances.

PUT validate/correlation/rule Validate a correlation rule.

Alerts

GET instance/{id}/alerts Get a list of alerts from an instance.

POST instance/{id}/alerts Acknowledge alerts.

GET instance/{id}/alertIds Get a list of alert IDs from an instance.

GET instance/{id}/alert/{alertID} Get the detailed summary of an alert.

GET instances/alert/severities Get a list of alert severities.

GET instances/alert/categories Get a list of alert categories.

POST instances/alert/fields Get a list of alert fields for the rule and sourceenvironment. The alert fields may be different dependingon the rule and source environment.

GET instance/{id}/alert/{alertID}/group/{groupName}/ eventKeys

Get a list of event references and sources. This supportsonly realtime instances.

19


For detailed information for each method and sample output, refer to the LogLogic Unity API onlinedocumentation using the following URL:

http://<hostname>:9682/docs

This assumes that the default port is used for this node. If you change the default port, update the URLaccordingly.

Event Correlation Language (ECL) ParameterThe REST API for creating instances of correlation requires an ECL ruleset parameter.

ECL Ruleset

The ECL ruleset is composed of different parts; mandatory and optional parameters. All keywords arecase insensitive.

The empty ruleset is as shown below:Begin Correlation Ruleset<identifier environment><Correlation Rule 1><Correlation Rule 2>…<Trigger 1><Trigger 2>…End Correlation Ruleset

After each run of the ruleset, alerts are generated on the Alerts page. An alert may be present severaltime if it has changed during its lifetime (consider the alertID fields to check whether or not two alertsare the same).

Rule

Rules contain additional information before the Blok content.

● a name● a uuid (optional)● a description (optional)● creation / modification users and dates (optional)Rule <identifier> [Disabled][ Uuid "<string>" ][ Description "<description text>" ][ Created By "<original author>" ][ Creation yyyy-MM-dd hh:mm:ss ][ Last Modified By "<last modifier>" ][ Last Modification yyyy-MM-dd hh:mm:ss ]

Triggers

Triggers syntax is as shown below:

Trigger <identifier> For Rule <rule identifier> [ Disabled ][ Description <string> ][ Created By <string> ][ Creation yyyy-MM-dd hh:mm:ss ][ Last Modified By <string> ][ Last Modification yyyy-MM-dd hh:mm:ss ]<action>

Action

Action syntax is as shown below:

20


Generate Alert[ Name <string> ][ Description <string> ][ Severity <string> ][ Category <string> ][ Max <integer> Alerts Per ( Day | Hour | Minute ) ]( <notification> )*

Notification

Notification syntax is as shown below:

Notify By Email ( To <string> )+ ( Cc <string> )*Subject <string>Body <string>[ Max <integer> Emails Per ( Day | Hour | Minute ) ]

Notify By Syslog To <hotname/IP>[:<port>][ Protocol ( Udp | Tcp ) ][ Delimiter <String> ][Facility <number>][Severity <number>]Message <string>[Max xxx Messages Per Minute ]

Correlation WorkflowUsing the Correlation APIs, you can create a new rule, test it on historic data, and then add it to a real-time instance so that it gets run on new, incoming data. Once it is running and generating alerts, youcan retrieve and acknowledge the alerts.

For parameter details, refer to the online LogLogic Unity API documentation.

A typical workflow is as follows:

1. Creating, Updating, and Deleting Replay Instance

a. Create a replay instance using a rule that is defined in ECL. For information, see EventCorrelation Language (ECL) Parameter.

b. Check the status of the instance to ensure it has finished processing the historic data.

c. Retrieve the correlation events to ensure the rule is operating as expected.

d. If the output is not as expected, update the instance with new ECL and view the new results.

e. Once the ECL is finalized, the replay instance should be deleted.

2. Creating or Updating Real-time Instance

You would either create a real-time instance or update the existing instance.

a. Create a real-time instance using the rule with a trigger added.

b. Check the status of the real-time instance.

c. Update the instance to include the new rule and trigger if necessary.

3. Retrieving and Acknowledging Alerts

a. Retrieve and view alerts

b. Acknowledge alerts

21


Creating, Updating, and Deleting Replay Instance

Create a replay instance

This method creates a new instance and runs it on the specified data.

RequestURL

POST http://localhost:9682/api/v1/instances

RequestBody

{ "instanceType": "replay", "ecl": "Begin Correlation Ruleset Rule Sample Use sample Within 1h Event Group LoginFailed At Least 1 Events Where ll_eventActionID = \"2\" And ll_eventStatusID = \"4\" With The Same ll_targetUser Having At Least 1 Distinct ll_sourceDomain Limit 1000 Limits 10000 Groups And 100000 Events Event Group LoginSuccess At Least 1 Events Where ll_eventActionID = \"2\" And ll_eventStatusID = \"3\" With The Same ll_targetUser Limits 10000 Groups And 100000 Events Correlation LoginFailed->ll_targetUser == LoginSuccess->ll_targetUser LoginSuccess After LoginFailed End Correlation Ruleset", "startDate": "2014-01-01 00:00:00", "endDate": "2015-01-01 00:00:00"}

22


ResponseBody

{ "id": "a02a3a08-c068-48e4-b654-0c03cac44a58", "resultSetId": "0", "columnDescriptors": [ { "name": "ruleName", "type": "STRING", "clickable": false, "hasStats": true }, { "name": "oldestEventTime", "type": "TIMESTAMP", "clickable": false, "hasStats": true }, { "name": "newestEventTime", "type": "TIMESTAMP", "clickable": false, "hasStats": true }, { "name": "lastModificationTime", "type": "TIMESTAMP", "clickable": false, "hasStats": true }, { "name": "eventCount", "type": "LONG", "clickable": false, "hasStats": true }, { "name": "eventGroupCount", "type": "LONG", "clickable": false, "hasStats": true }, { "name": "closed", "type": "BOOLEAN", "clickable": false, "hasStats": true }, { "name": "closingTime", "type": "TIMESTAMP", "clickable": false, "hasStats": true }, { "name": "eventId", "type": "STRING", "clickable": false, "hasStats": true }, { "name": "LoginFailed_eventGroupName", "type": "STRING", "clickable": false, "hasStats": true }, { "name": "LoginFailed_oldestEventTime", "type": "TIMESTAMP", "clickable": false, "hasStats": true },

23


{ "name": "LoginFailed_newestEventTime", "type": "TIMESTAMP", "clickable": false, "hasStats": true }, { "name": "LoginFailed_eventCount", "type": "LONG", "clickable": true, "hasStats": true }, { "name": "LoginFailed_ll_targetUser", "type": "STRING", "clickable": false, "hasStats": true }, { "name": "LoginFailed_Count_Distinct_ll_sourceDomain_Limit_1000", "type": "LONG", "clickable": false, "hasStats": true }, { "name": "LoginFailed_distinctValues_ll_sourceDomain", "type": "STRING", "clickable": false, "hasStats": true }, { "name": "LoginSuccess_eventGroupName", "type": "STRING", "clickable": false, "hasStats": true }, { "name": "LoginSuccess_oldestEventTime", "type": "TIMESTAMP", "clickable": false, "hasStats": true }, { "name": "LoginSuccess_newestEventTime", "type": "TIMESTAMP", "clickable": false, "hasStats": true }, { "name": "LoginSuccess_eventCount", "type": "LONG", "clickable": true, "hasStats": true }, { "name": "LoginSuccess_ll_targetUser", "type": "STRING", "clickable": false, "hasStats": true } ], "type": "Correlation"}

Check status of the instance

This method gets the status of the instance.

24


RequestURL

GET http://localhost:9682/api/v1/instance/a02a3a08-c068-48e4-b654-0c03cac44a58

RequestResponse

{ "instanceId": "a02a3a08-c068-48e4-b654-0c03cac44a58", "revision": 0, "instanceType": "replay", "type": "Replay", "userId": 1, "tenant": "tenant1", "timeField": null, "eventKeyField": null, "environment": tenant1.shared, "ecl": "Begin Correlation Ruleset Rule Sample Use sample Within 1h Event Group LoginFailed At Least 1 Events Where ll_eventActionID = \"2\" And ll_eventStatusID = \"4\" With The Same ll_targetUser Having At Least 1 Distinct ll_sourceDomain Limit 1000 Limits 10000 Groups And 100000 Events Event Group LoginSuccess At Least 1 Events Where ll_eventActionID = \"2\" And ll_eventStatusID = \"3\" With The Same ll_targetUser Limits 10000 Groups And 100000 Events Correlation LoginFailed->ll_targetUser == LoginSuccess->ll_targetUser LoginSuccess After LoginFailed End Correlation Ruleset", "eventStartTime": "2014-01-01 00:00:00", "eventEndTime": "2015-01-01 00:00:00", "timeCreated": 1425410043145, "status": "completed", "currentRefClock": 1391557440000, "eventsReceived": 9651, "alertsGenerated": 0, "errorCount": 0, "messages": null, "searchFilter": null, "timeFilter": null}

The "status": "completed" indicates that the instance has processed all the historic data in the system.

Retrieve correlation events

This method gets correlation events for the instance.

RequestURL

GET http://localhost:9682/api/v1/instance/a02a3a08-c068-48e4-b654-0c03cac44a58/correlationevents?from=0&size=10&type=MESSAGES

25


RequestResponse

{ "instanceId": "a02a3a08-c068-48e4-b654-0c03cac44a58", "resultSetId": null, "from": 0, "count": 6, "totalCount": 6, "eventsProcessed": 9651, "eventsTotalCount": 9651, "lastUpdate": 165454, "results": [ [ "Sample", 1391379513000, 1391379568000, 1391383180000, 2, 2, true, 1391383113000, "b1635223-45c9-4a86-986d-fc1e68afbd1c", "LoginFailed", 1391379513000, 1391379513000, 1, "-", "1", "{CORP}", "LoginSuccess", "1391379568000", 1391379568000, 1, "-" ], [ "Sample", 1391387647000, 1391389543000, 1391391492000, 8, 2, true, 1391391247000, "fd763245-a98b-4090-ab32-9d1f57a3f2d0", "LoginFailed", 1391387647000, 1391389543000, 7, "-", "1", "{CORP}", "LoginSuccess", "1391387772000", 1391387772000, 1, "-" ], [ "Sample", 1391412040000, 1391413542000, 1391415831000, 32, 2, true, 1391415640000, "a262e56d-715b-408d-b226-0e7a0d57ea91", "LoginFailed", 1391412040000, 1391412040000, 1,

26


"-", "1", "{CORP}", "LoginSuccess", "1391412144000", 1391413542000, 31, "-" ], [ "Sample", 1391428291000, 1391428309000, 1391431920000, 2, 2, true, 1391431891000, "e57b39a5-3d59-48e9-ada9-afe402ea7f24", "LoginFailed", 1391428291000, 1391428291000, 1, "-", "1", "{CORP}", "LoginSuccess", "1391428309000", 1391428309000, 1, "-" ], [ "Sample", 1391527459000, 1391527567000, 1391531129000, 4, 2, true, 1391531059000, "0fa2348f-7e5b-4c83-836b-eb351b63aba1", "LoginFailed", 1391527459000, 1391527459000, 1, "-", "1", "{corp}", "LoginSuccess", "1391527506000", 1391527567000, 3, "-" ], [ "Sample", 1391535571000, 1391535807000, 1391535692000, 225, 2, true, 1391539171000, "f81b2248-cba0-47c7-8dde-8b7cccd7df7a", "LoginFailed", 1391535571000, 1391535807000, 71, "portal",

27


"6", "{, AEPNGG, CORP, P3041589, P3050130, corp}", "LoginSuccess", "1391535571000", 1391535806000, 154, "portal" ] ], "statsData": null, "chartData": null, "messages": null}

The "totalCount": 6 indicates that the rule matched data 6 times and "count": 6 indicates that all 6 of thematches were returned.

Update the replay instance

You can update the instance. In this example, the ECL is changed so the time window for the eventgroups is 2 hours.

RequestURL

POST http://localhost:9682/api/v1/instance/a02a3a08-c068-48e4-b654-0c03cac44a58

RequestBody

{ "ecl": "Begin Correlation Ruleset Rule Sample Use sample Within 2h Event Group LoginFailed At Least 1 Events Where ll_eventActionID = \"2\" And ll_eventStatusID = \"4\" With The Same ll_targetUser Having At Least 1 Distinct ll_sourceDomain Limit 1000 Limits 10000 Groups And 100000 Events Event Group LoginSuccess At Least 1 Events Where ll_eventActionID = \"2\" And ll_eventStatusID = \"3\" With The Same ll_targetUser Limits 10000 Groups And 100000 Events Correlation LoginFailed->ll_targetUser == LoginSuccess->ll_targetUser LoginSuccess After LoginFailed End Correlation Ruleset", "startDate": "2014-01-01 00:00:00", "endDate": "2015-01-01 00:00:00"}

ResponseBody "a02a3a08-c068-48e4-b654-0c03cac44a58"

As a result of updating the instance, the modified rule will be applied to the data and a new set ofcorrelation events will be generated.

Delete the replay instance

You should delete the instance to free all resources used by the instance.

RequestURL

DELETE http://localhost:9682/api/v1/instance/a02a3a08-c068-48e4-b654-0c03cac44a58

ResponseBody true

28


Creating or Updating Real-time InstanceThe web application creates and manages its own real-time instance. It is not recommended to modifythis instance via the REST API. Instead, using the REST API you should create and manage your ownreal-time instances.

If you have not already created a real-time instance, follow the same process as described in the earliersection for creating the replay instance, except specify "instanceType": "realtime". There is no need tospecify a start or end date since the rules are only applied to new data entering into the system.

Real-time instances only process new data entering into the system, so once a real-time instance hasbeen created you will need to wait until new data arrives before any alerts can be generated.

Create a real-time instance

This method creates a new instance and runs it on new data entering the system.

RequestURL

POST http://localhost:9682/api/v1/instances

RequestBody

{ "instanceType": "realtime", "ecl": "Begin Correlation Ruleset Rule Sample Use sample Within 2h Event Group LoginFailed At Least 1 Events Where ll_eventActionID = \"2\" And ll_eventStatusID = \"4\" With The Same ll_targetUser Having At Least 1 Distinct ll_sourceDomain Limit 1000 Limits 10000 Groups And 100000 Events Event Group LoginSuccess At Least 1 Events Where ll_eventActionID = \"2\" And ll_eventStatusID = \"3\" With The Same ll_targetUser Limits 10000 Groups And 100000 Events Correlation LoginFailed->ll_targetUser == LoginSuccess->ll_targetUser LoginSuccess After LoginFailed Trigger [Sample] For Rule [Sample] Description \"\" Created By \"admin\" Creation 2015-03-04 13:16:57 Last Modified By \"admin\" Last Modification 2015-03-04 13:16:57 Generate Alert Name \"Sample\" Description \"Too many failed logins\" Severity \"high\" Category \"Suspicious Activity\" Max 100 Alerts Per hour End Correlation Ruleset" }

RequestResponse

{ "id": "e4f4751a-4a4e-4312-b38e-1413ead1e91d", "resultSetId": null, "columnDescriptors": null, "type": "Correlation"}

The "resultSetId" and "columnDescriptors" will be null when the instance is a real-time instance. If areal-time instance has already been created via the API, the new rule can be added to it by updating theECL as described in the Update Instance section. Once the data has been received, alerts are generated.

Check status of the instance

This method gets the status of the instance.

RequestURL

GET http://localhost:9682/api/v1/instance/e4f4751a-4a4e-4312-b38e-1413ead1e91d

29


RequestResponse

{ "instanceId": "e4f4751a-4a4e-4312-b38e-1413ead1e91d", "revision": 0, "instanceType": "realtime", "type": "RealTime", "userId": 1, "tenant": "tenant1", "timeField": null, "eventKeyField": null, "environment": null, "ecl": "Begin Correlation Ruleset Rule Sample Use sample Within 2h Event Group LoginFailed At Least 1 Events Where ll_eventActionID = \"2\" And ll_eventStatusID = \"4\" With The Same ll_targetUser Having At Least 1 Distinct ll_sourceDomain Limit 1000 Limits 10000 Groups And 100000 Events Event Group LoginSuccess At Least 1 Events Where ll_eventActionID = \"2\" And ll_eventStatusID = \"3\" With The Same ll_targetUser Limits 10000 Groups And 100000 Events Correlation LoginFailed->ll_targetUser == LoginSuccess->ll_targetUser LoginSuccess After LoginFailed Trigger [Sample] For Rule [Sample] Description \"\" Created By \"admin\" Creation 2015-03-04 13:16:57 Last Modified By \"admin\" Last Modification 2015-03-04 13:16:57 Generate Alert Name \"Sample\" Description \"Too many failed logins\" Severity \"high\" Category \"Suspicious Activity\" Max 100 Alerts Per hour End Correlation Ruleset", "eventStartTime": null, "eventEndTime": null, "timeCreated": 1425504222526, "status": "running", "currentRefClock": 1423089839000, "eventsReceived": 9932, "alertsGenerated": 4, "errorCount": 0, "messages": null, "searchFilter": null, "timeFilter": null}

The "status": "running" indicates that the instance is actively running and will receive any new data thatis added to the system. The "eventsReceived": "9932" indicates that 9932 log events have been receivedby this instance and the value of "alertsGenerated" indicates that 4 alerts have been generated.

Retrieving and Acknowledging AlertsYou can get a list of generated alerts for a specific instance. All alerts should be acknowledged toindicate that they have been dealt with.

Retrieve alerts

This method shows that 4 alerts can be retrieved.

RequestURL

GET http://localhost:9682/api/v1/instance/e4f4751a-4a4e-4312-b38e-1413ead1e91d/alerts

30


ResponseBody

{ "totalCount": 4, "rows": [ { "instanceId": "e4f4751a-4a4e-4312-b38e-1413ead1e91d", "instanceRev": 0, "alertId": "f65b866e-cd0d-471f-a3e3-cceaa13dfadf", "correlationEventId": "78467e44-fdad-4c32-8a6e-25a63980b30e", "ruleName": "Sample", "ruleUuid": null, "severity": "high", "alertName": "Sample", "category": "Suspicious Activity", "description": "Too many failed logins", "detectionTime": 1423070713000, "alertTime": 1423070713000, "oldestEventTime": 1423063459000, "newestEventTime": 1423068165000, "lastModificationTime": 1423070713000, "slaTime": 1423074313000, "timeToExpiration": -2430514181, "eventCount": 1262, "acknowledged": false }, { "instanceId": "e4f4751a-4a4e-4312-b38e-1413ead1e91d", "instanceRev": 0, "alertId": "d948234d-9ebc-4943-9d1e-5eb91cd38732", "correlationEventId": "e6e4ed48-64d4-42ae-9ee5-b2e91df2b40e", "ruleName": "Sample", "ruleUuid": null, "severity": "high", "alertName": "Sample", "category": "Suspicious Activity", "description": "Too many failed logins", "detectionTime": 1422971551000, "alertTime": 1422971623000, "oldestEventTime": 1422964291000, "newestEventTime": 1422965068000, "lastModificationTime": 1422971551000, "slaTime": 1422982800000, "timeToExpiration": -2522027181, "eventCount": 18, "acknowledged": false }, { "instanceId": "e4f4751a-4a4e-4312-b38e-1413ead1e91d", "instanceRev": 0, "alertId": "7a1a3dc0-f77b-4980-a679-d35c30f2e88d", "correlationEventId": "ffcf701b-e17a-4a9b-a6a7-1278ecf4287e", "ruleName": "Sample", "ruleUuid": null, "severity": "high", "alertName": "Sample", "category": "Suspicious Activity", "description": "Too many failed logins", "detectionTime": 1422930970000, "alertTime": 1422958724000, "oldestEventTime": 1422923647000, "newestEventTime": 1422925543000, "lastModificationTime": 1422930970000, "slaTime": 1422982800000, "timeToExpiration": -2522027181, "eventCount": 8, "acknowledged": false }, { "instanceId": "e4f4751a-4a4e-4312-b38e-1413ead1e91d", "instanceRev": 0, "alertId": "7682adf8-d647-48eb-b740-f7a30104a996",

31


"correlationEventId": "b11d9e38-0c98-48be-a4e5-8bd6f10274c1", "ruleName": "Sample", "ruleUuid": null, "severity": "high", "alertName": "Sample", "category": "Suspicious Activity", "description": "Too many failed logins", "detectionTime": 1422922813000, "alertTime": 1422922813000, "oldestEventTime": 1422915513000, "newestEventTime": 1422920158000, "lastModificationTime": 1422922813000, "slaTime": 1422926413000, "timeToExpiration": -2578414181, "eventCount": 102, "acknowledged": false } ]}

Alert details

The details for a given alert can then be retrieved. The details include the contents of the event groupsand the log events that contributed to the alert being generated.

RequestURL

GET http://localhost:9682/api/v1/instance/e4f4751a-4a4e-4312-b38e-1413ead1e91d/alert/7682adf8-d647-48eb-b740-f7a30104a996

32


RequestResponse

{ "instanceId": "e4f4751a-4a4e-4312-b38e-1413ead1e91d", "alertId": "7682adf8-d647-48eb-b740-f7a30104a996", "ruleName": "Sample", "rule": "Rule Sample\r\n\tUse sample\r\n\tWithin 2h\r\n\tEvent Group LoginFailed\r\n\t\tAt Least 1 Events\r\n\t\tWhere ll_eventActionID = \"2\" And ll_eventStatusID = \"4\"\r\n\t\tWith The Same ll_targetUser\r\n\t\tHaving At Least 1 Distinct ll_sourceDomain Limit 1000\r\n\t\tLimits 10000 Groups And 100000 Events\r\n\tEvent Group LoginSuccess\r\n\t\tAt Least 1 Events\r\n\t\tWhere ll_eventActionID = \"2\" And ll_eventStatusID = \"3\"\r\n\t\tWith The Same ll_targetUser\r\n\t\tLimits 10000 Groups And 100000 Events\r\n\tCorrelation\r\n\t\tLoginFailed->ll_targetUser == LoginSuccess->ll_targetUser\r\n\t\tLoginSuccess After LoginFailed\r\n\r\n", "time": "Mon Feb 02 16:20:13 PST 2015", "alertName": "Sample", "description": "Too many failed logins", "severity": "high", "category": "Suspicious Activity", "alertTime": "Mon Feb 02 16:20:13 PST 2015", "oldestEventTime": "Mon Feb 02 14:18:33 PST 2015", "newestEventTime": "Mon Feb 02 15:35:58 PST 2015", "slaTime": 1422926413000, "timeToExpiration": -2579255911, "eventCount": 102, "values": [ { "key": "eventId", "value": "b11d9e38-0c98-48be-a4e5-8bd6f10274c1", "dataType": "STRING" }, { "key": "ruleName", "value": "Sample", "dataType": "STRING" }, { "key": "lastModificationTime", "value": "Mon Feb 02 16:20:13 PST 2015", "dataType": "TIME" } ], "eventGroups": [ { "name": "LoginSuccess", "oldestEventTime": 1422915568000, "newestEventTime": 1422920158000, "eventCount": 101, "values": [ { "key": "sys_eventTime", "value": "Mon Feb 02 14:19:28 PST 2015", "dataType": "TIME" }, { "key": "ll_targetUser", "value": "-", "dataType": "STRING" } ], "eventReferences": [ "tenant1|shared|1|129", "tenant1|shared|1|130", "tenant1|shared|1|131", "tenant1|shared|1|132", "tenant1|shared|1|133", "tenant1|shared|1|134", "tenant1|shared|1|135", "tenant1|shared|1|136", "tenant1|shared|1|137",

33


"tenant1|shared|1|210", "tenant1|shared|1|211", "tenant1|shared|1|212", "tenant1|shared|1|213", "tenant1|shared|1|214", "tenant1|shared|1|215", "tenant1|shared|1|216", "tenant1|shared|1|217", "tenant1|shared|1|218", "tenant1|shared|1|219", "tenant1|shared|1|220", "tenant1|shared|1|221", "tenant1|shared|1|222", "tenant1|shared|1|223", "tenant1|shared|1|224", "tenant1|shared|1|225", "tenant1|shared|1|226", "tenant1|shared|1|228", "tenant1|shared|1|229", "tenant1|shared|1|230", "tenant1|shared|1|231", "tenant1|shared|1|232" ] }, { "name": "LoginFailed", "oldestEventTime": 1422915513000, "newestEventTime": 1422915513000, "eventCount": 1, "values": [ { "key": "Count(Distinct ll_sourceDomain Limit 1000)", "value": "1", "dataType": "LONG" }, { "key": "sys_eventTime", "value": "Mon Feb 02 14:18:33 PST 2015", "dataType": "TIME" }, { "key": "distinctValues(ll_sourceDomain)", "value": "{CORP}", "dataType": "STRING" }, { "key": "ll_targetUser", "value": "-", "dataType": "STRING" } ], "eventReferences": [ "tenant1|shared|1|128" ] } ], "acknowledgements": [], "name": "Sample"}

Acknowledge an alert

Each alert should be acknowledged to indicate that it has been dealt with. More than one alert can beacknowledged with one request by supplying a list of alert IDs. The category and severity of the alertscan also be changed as part of the acknowledgment request.

This method acknowledges an alert.

35


RequestURL

POST http://localhost:9682/api/v1/instance/e4f4751a-4a4e-4312-b38e-1413ead1e91d/alerts

RequestBody

{ "alertIds": [ "d948234d-9ebc-4943-9d1e-5eb91cd38732", "7a1a3dc0-f77b-4980-a679-d35c30f2e88d" ], "category": "False positive", "severity": "info", "comment": "False alarm"}

RequestResponse

true

The possible values for "category" and "severity" are returned by the following calls.

RequestURL

GET http://localhost:9682/api/v1/instances/alert/categories

RequestResponse

[ "Attack on third party", "Authorized Activity", "Authorized security testing", "Emergency changes", "False positive", "Known error", "LogLogic Event", "Network Noise", "Security Alert", "Suspicious Activity", "Unauthorized Activity", "Unknown"]

RequestURL

GET http://localhost:9682/api/v1/instances/alert/severities

RequestResponse

[ { "severityName": "info", "slaDuration": 1440 }, { "severityName": "low", "slaDuration": 720 }, { "severityName": "medium", "slaDuration": 240 }, { "severityName": "high", "slaDuration": 60 }, { "severityName": "none", "slaDuration": 10 }]

36


tibco loglogic unity developer's guide · 2015. 12. 3. · tibco loglogic® unity...

Documents