tibco loglogic unity user's guide · 2016-06-02 · tibco loglogic® unity user's guide...

TIBCO LogLogic® Unity User's GuideSoftware Release 2.6June 2016

Two-Second Advantage®

Important Information

SOME TIBCO SOFTWARE EMBEDS OR BUNDLES OTHER TIBCO SOFTWARE. USE OF SUCHEMBEDDED OR BUNDLED TIBCO SOFTWARE IS SOLELY TO ENABLE THE FUNCTIONALITY (ORPROVIDE LIMITED ADD-ON FUNCTIONALITY) OF THE LICENSED TIBCO SOFTWARE. THEEMBEDDED OR BUNDLED SOFTWARE IS NOT LICENSED TO BE USED OR ACCESSED BY ANYOTHER TIBCO SOFTWARE OR FOR ANY OTHER PURPOSE.

USE OF TIBCO SOFTWARE AND THIS DOCUMENT IS SUBJECT TO THE TERMS ANDCONDITIONS OF A LICENSE AGREEMENT FOUND IN EITHER A SEPARATELY EXECUTEDSOFTWARE LICENSE AGREEMENT, OR, IF THERE IS NO SUCH SEPARATE AGREEMENT, THECLICKWRAP END USER LICENSE AGREEMENT WHICH IS DISPLAYED DURING DOWNLOADOR INSTALLATION OF THE SOFTWARE (AND WHICH IS DUPLICATED IN THE LICENSE FILE)OR IF THERE IS NO SUCH SOFTWARE LICENSE AGREEMENT OR CLICKWRAP END USERLICENSE AGREEMENT, THE LICENSE(S) LOCATED IN THE “LICENSE” FILE(S) OF THESOFTWARE. USE OF THIS DOCUMENT IS SUBJECT TO THOSE TERMS AND CONDITIONS, ANDYOUR USE HEREOF SHALL CONSTITUTE ACCEPTANCE OF AND AN AGREEMENT TO BEBOUND BY THE SAME.

This document contains confidential information that is subject to U.S. and international copyright lawsand treaties. No part of this document may be reproduced in any form without the writtenauthorization of TIBCO Software Inc.

TIBCO, LogLogic, and Two-Second Advantage are either registered trademarks or trademarks ofTIBCO Software Inc. in the United States and/or other countries.

Enterprise Java Beans (EJB), Java Platform Enterprise Edition (Java EE), Java 2 Platform EnterpriseEdition (J2EE), and all Java-based trademarks and logos are trademarks or registered trademarks ofOracle Corporation in the U.S. and other countries.

All other product and company names and marks mentioned in this document are the property of theirrespective owners and are mentioned for identification purposes only.

THIS SOFTWARE MAY BE AVAILABLE ON MULTIPLE OPERATING SYSTEMS. HOWEVER, NOTALL OPERATING SYSTEM PLATFORMS FOR A SPECIFIC SOFTWARE VERSION ARE RELEASEDAT THE SAME TIME. SEE THE README FILE FOR THE AVAILABILITY OF THIS SOFTWAREVERSION ON A SPECIFIC OPERATING SYSTEM PLATFORM.

THIS DOCUMENT IS PROVIDED “AS IS” WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSOR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OFMERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT.

THIS DOCUMENT COULD INCLUDE TECHNICAL INACCURACIES OR TYPOGRAPHICALERRORS. CHANGES ARE PERIODICALLY ADDED TO THE INFORMATION HEREIN; THESECHANGES WILL BE INCORPORATED IN NEW EDITIONS OF THIS DOCUMENT. TIBCOSOFTWARE INC. MAY MAKE IMPROVEMENTS AND/OR CHANGES IN THE PRODUCT(S)AND/OR THE PROGRAM(S) DESCRIBED IN THIS DOCUMENT AT ANY TIME.

THE CONTENTS OF THIS DOCUMENT MAY BE MODIFIED AND/OR QUALIFIED, DIRECTLY ORINDIRECTLY, BY OTHER DOCUMENTATION WHICH ACCOMPANIES THIS SOFTWARE,INCLUDING BUT NOT LIMITED TO ANY RELEASE NOTES AND "READ ME" FILES.

Copyright © 2014-2016 TIBCO Software Inc. All rights reserved.

TIBCO Software Inc. Confidential Information

2

TIBCO LogLogic® Unity User's Guide

Contents

TIBCO Documentation and Support Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Signing into the Web UI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10

Signing out of the Web UI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11

Editing your Profile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12

Import File Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Getting Data into LogLogic Unity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14

Canceling an Import . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Search Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Using Content Assist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Using the Search field . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

Optimizing Search Queries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17

Using the Time field . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Search Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19

Charts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19

Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21

About Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

Raw Data Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

Table Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .30

Correlation Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

Search Syntax Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

Event Query Language Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

Common Search Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .37

USE Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

FILTER Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

Predefined Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

Time Range Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

COLUMNS Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

GROUP BY Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

SORT BY Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

LIMIT Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

Search Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

Event Correlation Language Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

Rule Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

Identifier Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

3


Event Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .57

Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

Aggregation Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

Having Clause . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

Correlation Criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

Correlation Blok (ECL) Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .64

About Bloks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

Filter Bloks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

Correlation Bloks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

Viewing All Bloks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

Adding a Blok . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

Modifying Bloks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

Deleting Bloks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .69

Time Bloks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

Viewing All Time Bloks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

Adding a Time Blok . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

Modifying Time Bloks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

Source Bloks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

Manage Bloks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

Manage Dashboard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .74

Viewing Dashboard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .75

Adding Widgets to a Dashboard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

Line Widget . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .76

Bar Widget . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

Pie Widget . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .79

Number Widget . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

Gauge Widget . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

Stacked Column Widget . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .82

Combined Widget . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .83

Editing a Widget . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .85

Deleting a Widget . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

Duplicating a Dashboard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

Deleting a Dashboard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

Administration Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

Manage Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .89

Manage Triggers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

Viewing Triggers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

Adding a Trigger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

4


Configuring SMTP Connection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .92

Editing Triggers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

Synchronizing Triggers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

Enabling or Disabling Triggers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

Deleting Triggers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .94

Manage Forwarding Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

Viewing Forwarding Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .94

Adding a Forwarding Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .95

Editing a Forwarding Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

Enabling or Disabling Forwarding Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

Deleting a Forwarding Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

Manage Aggregation Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

Viewing Aggregation Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .98

Adding an Aggregation Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .99

Editing an Aggregation Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

Deleting an Aggregation Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

Enabling or Disabling Aggregation Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

Monitor Alerts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

Viewing Alerts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .102

Acknowledging Alerts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .104

Viewing Alert Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

Viewing Event Group Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

Manage Source Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

Viewing Source Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

Adding a Source Configuration in Graphical Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

Defining a Source Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

Adding a Parsing Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

Editing Parsing Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114

Defining Parsing Rules Order . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

Copying Parsing Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .115

Deleting Parsing Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .115

Managing Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

Adding a Source Configuration in Raw Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .117

Enabling or Disabling Source Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

Editing Source Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

Duplicating Source Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

Deleting Source Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .119

Manage Sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

Viewing Sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

5


Setting up the Export Interval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .122

Manage Domains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

Viewing Domains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .123

Adding a Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .124

Editing a Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .125

Deleting a Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

Manage Data Nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

Viewing Data Nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .126

Manage Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

Viewing Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .127

Adding a User . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .128

Editing User Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .129

Disabling a User . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

Deleting User . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

Enabling LogLogic LMI Users to Access LogLogic Unity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .130

Setting up Remote Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

Manage Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

Viewing Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

Adding a Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

Editing a Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

Disabling a Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

Deleting a Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

Manage System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .135

Manage Retention Policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

Viewing Retention Policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

Adding a Retention Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .136

Editing a Retention Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136

Deleting a Retention Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136

Manage Collectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

Viewing Collectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

Adding a Hawk Collector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

Using TIBCO Rendezvous (RV) Message Transport . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140

Using TIBCO Enterprise Message Service (EMS) Message Transport . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140

Adding a Syslog Collector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

Adding a BusinessWorks Metrics Collector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143

Editing a Collector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .144

Enabling or Disabling Collectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144

Deleting a Collector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144

Manage Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146

6


Configuring a Reports Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .146

Viewing Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147

Running a Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147

Supported Regular Expression Characters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .149

Supported Log Sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151

Filter Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

Parsing Rule JSON syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

7


TIBCO Documentation and Support Services

Documentation for this and other TIBCO products is available on the TIBCO Documentation site. Thissite is updated more frequently than any documentation that might be included with the product. Toensure that you are accessing the latest available help topics, please visit:

https://docs.tibco.com

Product-Specific Documentation

Documentation for TIBCO products is not bundled with the software. Instead, it is available on theTIBCO Documentation site. To directly access documentation for this product, double-click thefollowing file:

TIBCO_HOME/release_notes/TIB_logu_version_docinfo.html

where TIBCO_HOME is the top-level directory in which TIBCO products are installed. On Windows,the default TIBCO_HOME is C:\tibco. On UNIX systems, the default TIBCO_HOME is /opt/tibco.The following documents for this product can be found in the TIBCO Documentation site:

● TIBCO LogLogic® Unity Installation and Configuration

● TIBCO LogLogic® Unity User's Guide

● TIBCO LogLogic® Unity Developer's Guide

● TIBCO LogLogic® Unity Tutorials

How to Contact TIBCO Support

For comments or problems with this manual or the software it addresses, contact TIBCO Support:

● For an overview of TIBCO Support, and information about getting started with TIBCO Support,visit this site:

http://www.tibco.com/services/support

● If you already have a valid maintenance or support contract, visit this site:

https://support.tibco.com

Entry to this site requires a user name and password. If you do not have a user name, you canrequest one.

How to Join TIBCOmmunity

TIBCOmmunity is an online destination for TIBCO customers, partners, and resident experts. It is aplace to share and access the collective experience of the TIBCO community. TIBCOmmunity offersforums, blogs, and access to a variety of resources. To register, go to the following web address:

https://www.tibcommunity.com

8


https://docs.tibco.com

http://www.tibco.com/services/support

https://support.tibco.com

https://www.tibcommunity.com

Overview

LogLogic® Unity is a sleek, modern, and scalable platform enabling technical teams to resolve openissues that require advanced troubleshooting techniques, complex root cause analysis, or deepforensics.

LogLogic Unity is a Log processing Search & Alerting tool that takes data from any source andstructures that data. This allows for intuitive, fast, and complete interaction with data, resulting infaster turnaround from open to close in issue resolution. Its powerful Web User Interface (UI) enablesfast and flexible searching, correlation, and alerting. This provides operational insights intoinfrastructure and application performance and security events.

Key Features

● Modular search queries: Use all or part of saved search filters to build new search queries using newbuilding Blok technology.

● Multiple search queries: Run multiple searches at the same time.

● Working data sets: Work with multiple search results without losing what you are working on. Walkaway and come back without losing your search results.

● Data lookup: Enrich your experience with lookup tables enhancing search and alerting capabilities.

● Data at rest correlation: Perform advanced correlation against historical data to identify trends.

● Data in motion correlation: Maintain advanced correlation in memory to identify key patterns foralerting.

● Comprehensive APIs: Leverage core functionality using intuitive APIs built on RepresentationalState Transfer (REST).

● Scalable clustering technology: Scale horizontally as needed to maintain performance and storage.

The LogLogic Unity architectural view is shown in the following illustration:

9


Signing into the Web UI

The Web UI enables fast and flexible searching, correlation, and alerting.

Prerequisites

You must start the LogLogic Unity system before logging into the Web UI. For details, refer to theTIBCO LogLogic® Unity Installation and Configuration guide.

Procedure

1. Open a browser and navigate to the URL http://localhost:9680, where localhost is the defaulthost name and 9680 is the default port number.

2. Enter your credentials.The default user name is admin and the default password is admin.

3. Click Sign in.On successful authentication for the first time user, the product walk-through screens are displayed.Click Next to continue. Click Try it out to open the Search tab and run a sample query using thesample source configuration.

The Search tab opens showing the sample query in the Search field. Click to view results of

the sample query. Click to start a new search. For details, see Search Basics.

10


Signing out of the Web UI

Procedure

1. Click located in the upper-right corner on the main header.

2. Click the Sign out link.When you are successfully signed out of the system, the Sign in window is displayed again.

11


Editing your Profile

You can update your own profile at any time.

Procedure

1. Click located in the upper-right corner on the main header, and select the Edit profile link.

2. Update the information in the corresponding fields.You can update the email address, personal information, phone number, and password. Only anadmin (a user with administrator privileges) can update the user ID after it is created.

3. If you update the password in the Old password field, enter the same password in the Newpassword field.

4. Click Save to save the updated information.After updating the information, the Sign in window is displayed. You must sign in using theupdated credentials.

12


Import File Data

You can import text data files directly into LogLogic Unity. If you add new data to an existing file, andthen re-import it, LogLogic Unity will re-index the entire file, not just the new data in the file. This canresult in a duplication of events.

Importing files into LogLogic Unity is a three step process:

1. Select a file from your machine to be uploaded. You can see a preview of your data, as parsed by thesystem source configuration.

2. Select how you want your data to look. You can apply a different source type to see whether thatoffers better results.

3. Import your selected file into LogLogic Unity.

When the file is fully imported, you can use the Search tab to easily interact with your data. Using theSearch tab, you can run simple and complex searches, save search elements and time ranges in the formof Bloks, and retrieve results to analyze failures or other anomalies.

Only file data that is on the browsers machine can be imported. After the raw data from the log file hasbeen imported, the imported data will be stored in the LogLogic Unity database.

The Import events from file window information is described below:

Field Description

File Name of selected file.

Raw events LogLogic Unity reads and displays the first 10 lines of data from yourselected file.

Source configuration Select from a list of pre-configured sources or define your own eventsource. The default value is system. You can add a new sourceconfiguration. For details, see Adding a Source Configuration inGraphical Mode.

The source configuration determines how LogLogic Unity will parseyour data. The purpose of source configuration is to help you applythe right source type to your incoming data. LogLogic Unity comeswith a large number of predefined source configurations. The sourcetype determines how LogLogic Unity formats your data duringparsing. By assigning the correct source type to your data, the datawill be parsed appropriately.

Timestamp format You can type in or select a string that represents the timestamppattern you expect LogLogic Unity to use when parsing your data.

LogLogic Unity will attempt to identify the format of the timestampin your data. If it fails, or the identified pattern is incorrect, you canselect a different timestamp, or type in your own.

Year You can also enter a four-digit year to replace the year portion of thedata. If the data in the log file spans multiple years, the year field willbe used as the starting year and incremented by 1 every time a yearboundary is detected.

13


Field Description

Time zone The time zone that you want to use for your data. The default value isthe current time zone used by LogLogic Unity.

Domain Select from a list of pre-configured domains or define a new domain.The default is set to shared. The three pre-defined domains are: shared,internal, and samples.

Create new domain You can create a new domain. For information on how to create a newdomain, see Adding a Domain.

Source IP Source IP address of the appliance your data is located on. The defaultvalue is 127.0.0.1.

Source type This field is populated based on your Source configuration selection.If the Source configuration you have selected has no type IDassociated with it, you can enter your own. The valid range optionsare from 16,384 to 65,535.

Preview parsed data LogLogic Unity displays 10 lines of parsed data based on yourselections. How it is displayed here is how it will be parsed with theselected source configuration. Review the preview to ensure data willbe displayed in the way you want. You can change your selectionsuntil you achieve the desired results.

Getting Data into LogLogic UnityYou can import text data files directly into LogLogic Unity.

Procedure

1. Click the Import button located on the upper-right corner of the page.

2. From the Upload and import file drop-down menu, click Choose file.

3. Import data into LogLogic Unity by browsing to and selecting the file you want to import, andselect Open.The Imports events from file window is displayed.

4. Select your Source configuration from the drop-down menu. This selection will populate theSource type field. You can add a new source configuration. For details, see Adding a SourceConfiguration in Graphical Mode.

5. Select a Timestamp pattern.If the suggested one is not correct, you can also enter a four-digit year that represents the year of thefirst log event in the file.

6. Enter a four-digit year to replace the year portion of the data in the Year field. If the data in the logfile spans multiple years, the year field will be used as the starting year and incremented by 1 everytime a year boundary is detected.

7. Select your Time zone.

8. Select the Domain from the drop-down menu. You can create a new domain. For information onhow to create a new domain, see Adding a Domain. The default domain is set to shared.

9. Enter the Source IP address.

14


10. When you are satisfied with your selections, click Import.Once the Import button is selected, you can monitor the status of your import from the Upload andimport file drop-down menu.

Canceling an ImportYou can cancel an import only while the file is being uploaded to LogLogic Unity.

Procedure

1. Click the Import button located on the upper-right corner of the page.The Importing files window is displayed.

2. Click the x located next to the file you want to stop importing.

15


Search Basics

From the Search tab, you can easily interact with your data. You can run simple and complex searches,save search elements and time ranges in the form of Bloks, and retrieve results to analyze failures orother anomalies.

The basic search retrieves all events that match the search term. Advanced searches are retrieved by a"pipeline" concept, where expressions are separated by pipes ("|"). LogLogic Unity search querylanguage is intuitive and efficient, lets you search large data and view results in seconds. The searchquery mainly supports three types of languages: Structured Query Language (SQL) dialect, EventQuery Language (EQL) and the Event Correlation Language (ECL). The Search Syntax Reference helpsyou understand how to form a search query. By default, results are returned in ascending order.

The Search and Time fields can be combined (AND-ed) or used alone as described below:

● If you define the time in the Search or Time field, the results are retrieved for the specified timeperiod.

● If you define the time in the Search field and Time field, the results are retrieved for the intersectionof the time periods.

● You must specify time in either in the Search or Time field.

All dates and times are defined in the local time zone of the machine where the system is installed.They are not based on the browser time zone.

Another kind of search query retrieves information records about LogLogic Unity itself, for example, itsconfiguration. These queries are referred as infrastructure queries. The basic search queries andinfrastructure queries are supported in the same way, except where indicated. The general differencesare:

● Internal information records are not necessarily related to log events and do not typically admit anevent time-stamp column. As a result, a Time field specification is not meaningful withininfrastructure queries, and their results have no default ordering.

● Whenever the documentation refers to events, that is meant as "records" when applied toinfrastructure queries.

For complex queries, you can create different types of Bloks that can be reused in future searches. Fordetailed information about how to build and use Bloks, see About Bloks.

For sample search examples, see Search Examples.

Click to add multiple search tabs. You can run multiple searches using different search elements onthe same data to analyze any anomalies.

Using Content AssistThe Content Assist feature shows typeahead or contextual matches and completions for each keywordas you type in the Search field. These contextual matches are retrieved from your data. You can getassistance for language syntax, column names, source configuration names, recent search history, andBlok names.

To enable the Content Assist feature, click located in the upper-right corner on the main header andselect the Enable Content Assist link. A check mark indicates that the content assist feature is enabled.

As you start typing in the Search field, the Content Assist panel is displayed:

● Suggestions help you build your search query by suggesting the next matching term.

● Matching terms identify the matching word as you continue to type.

16


● Sources lets you to define a source configuration to be used in your query.

● History displays all recent search entries that you can choose from to run a query.

Click on the term to select and add it in the Search field. Once you finish adding all terms, select thefragment in the Search field, and the Save fragment as Blok button is enabled. Click Save fragment asBlok to save the statement as a Blok for later use. The Add new Blok window opens. For instructionson how to add a Blok, see Adding a Blok.

Using the Search fieldYou can enter any valid combinations of syntax languages (SQL or EQL or ECL) with source, filter, orregular expressions. You can use single or multiple terms.

Enter USE to start an EQL statement and SELECT to start an SQL statement. You can search based onBloks. For details on how to add a new Blok or use the existing Blok, see About Bloks.

As you start typing, the Content Assist feature shows contextual matches and completions for each

keyword into the Search field. If the search expression syntax is valid, a green check mark is

displayed next to the syntax. Click to view results.

When copying a query from another rich text format application, such as Microsoft Word, intoLogLogic Unity can interfere with processing of the query. For example, extraneous characters can beadded to the query or straight quotation marks (") can be replaced with curly quotation marks (”) , like“ and ”, which are not part of a correct query string. Therefore, when copying from a rich formatsource, review the search query syntax and correct any errors before proceeding.

For example, enter the use system | sys_eventTime in -1d:NOW in the Search field to retrieveevents from the system source configuration profile within a certain time range.

Optimizing Search QueriesYou can optimize the performance of aggregation queries. The system pre-computes the aggregationsas LogLogic Unity events arrive in the system. As time progresses, the pre-computed aggregatesaccumulate providing results much faster than the queries that were not optimized. Such optimizedaggregation queries can be vital for creating responsive dashboards.

1. In the Search field, enter a regular EQL or SQL query that contains a GROUP BY statement andaggregated projections. For details, see GROUP BY Statement.

For example, use system | GROUP BY sys_tenant, sys_domain, weeks(sys_eventTime),days(sys_eventTime), hours(sys_eventTime) COLUMNS max(sys_bodySize),

avg(sys_offset)

2. After you click the button, an Optimize icon is displayed on the right side in the Searchfield for all aggregation queries.

This allows you to create an aggregation rule to optimize this query going forward.

17


3. Click the Optimize icon to add a rule using the Add Aggregation page.

For details about how to add an aggregation rule, see Adding an Aggregation Rule.

Using the Time fieldYou can enter absolute and relative time ranges. You can search based on Bloks.

From the Search tab, enter the time period in the Time field and click . For details on how toadd a new time Blok or use the existing Blok, see About Time Bloks.

All dates and times are defined in the local time zone of the machine where the system is installed andit is not based on the browser time zone.

For example, enter -5h to retrieve all events that occur in the last 5 hours.

The Time field must be empty when entering an infrastructure search queries. An example of invalidinfrastructure search query is: use Unity_config_bloks | sys_eventTime in -5d

18


Search Results

After running a search query, you can view search results in the Result tab.

You can visualize results using Charts or Data panel. After running a query, if you retrieve lots ofresults, you can group the results without having to issue a new query, and then drill-down into theinformation. You can see both aggregated counts as well as create visualization elements to betterisolate trends and issues. You can include multiple filters to narrow your results. Create a filter in thecontext of an event, and view results based on a specific filter.

After running the search query, a progress bar is displayed above the Result tab showing the number ofprocessed events. Based on your data, it might take a few minutes to retrieve results into all threepanels.

By default, a maximum of 10,000 results will be displayed in the Result tab. To increase the limit, usethe LIMIT clause in your query. See LIMIT Statement for details.

Add multiple result tabs to view the same data in different forms. Click to add multiple result tabs.When results are grouped together, a new Result tab is displayed showing the grouped results for theselected value.

The Result tab is divided into three panels:

● Charts display results graphically using a line chart in the top panel.

● Columns provide all available columns and their associated values based on each search query inthe left bottom panel.

● Data display data in different formats in the right bottom panel: raw format and normalized tabularformat.

ChartsA chart is a visual representation of your data. By using elements such as lines (in a line chart), a chartdisplays a series of numeric data in a graphical format.

You can add multiple result tabs to view the same data in different formats. A chart displays the totalevent count for a specified time period. You can use different options to view chart details, zoom in andout of the chart, and show or hide chart panel.

Charts are not supported for infrastructure queries.

From the Charts panel, you can perform the following tasks:

19


● Show or hide Charts panel

Click the icon located in the upper-right corner of the Charts panel to show the charts panel.

Click to hide the Charts panel.

● Zoom in or out of Charts

You can zoom in or zoom out of a particular area of chart using the time-range picker.

Grab the handles on the X-axis time-range picker, it turns into a slider. Drag the slider across the X-axis to define the time range that you want to zoom in. A chart is updated for the selected time.

The following line chart displays the zoomed in data for a specified time range and the Data panelshows the filtered results for the corresponding time range.

You can expand and collapse the time range by dragging the borders of the selected time range tothe desired location. Once you define the time range, position the mouse inside the selected timerange and drag the slider to define the new time range. Similarly, you can define a specific time byclicking on the chart. The time range can be adjusted at any time.

As you adjust the time range on the chart, the Columns and the Data panels are adjustedautomatically for the selected time range.

● View Chart details

Hover your mouse over a certain area of the chart to view the details.

20


● Filter results based on the time range

You can fine-tune your search results based on the time range. Click the event count (the line thatrepresents the number of events) on the chart or define the time range by zooming in on the chart toview results in the Data panel. A new filter is added for the defined time and the filtered results aredisplayed on the Data panel.

ColumnsBased on your search query, all available columns are displayed in the Columns panel. You can grouptogether your results based on any column and the value associated with that column. Similarly,filtering helps you fine-tune your search results when analyzing big data.

By default, three system columns are displayed to show results in the Data panel. For a list of systemcolumns, see About Columns.

From the Columns panel, you can perform the following tasks:

● Show or hide Columns panel

Click located on the right corner to hide the Columns panel. Click to show the Columnspanel.

● Find columns

You can quickly find the desired column by typing the column name in the Find field. As you starttyping a column name in the Find field, all possible columns that start with the letters that are typedget displayed in the pane. The Columns panel is refreshed based on the selection.

● Show or hide columns from the Data panel

21


Select the check box to show the column in the Data panel. Clear the check box to hide the columnfrom the Data panel. Click Select all to select all columns. Click Deselect all to hide all columns.

The located on the left side of the column name defines that the column is displayed in the Datapanel. The Data panel gets updated immediately based on your selection.

● View column value details

Click the column value and then select Show values to view the details of the selected value. Thewindow displays a maximum 100 distinct values for the selected column. The Percent column iscalculated using the maximum 100 distinct values. When the distinct values for a column exceeds100, the Percent column is not displayed. If you filter on a particular column value, then the percentvalue on the top shows the percentage of occurrence of this particular column value in the entireresult set.

The following illustration displays values for the column sys_bodySize.

● Filter results based on the column value

Click the Value link and select Include this filter to filter results based on that value. If you selectthe Exclude this filter option, the results are displayed without the defined value. You can addmultiple filters. Select Remove this filter to remove the selected filter from the results. The blue icon

represents included values and red icon represents excluded values from filtering data on theData panel.

The following illustration displays filtered results based on the value filter 610 included for thecolumn sys_bodySize.

22


● Edit value filters to refine results

Based on your selection, a new filter is added in the Data panel and the refined results are displayedbased on the filter. Click inside the value filter box to edit the value. Click the check mark to updatethe value changes. The Data panel results are refreshed based on the updated filters.

When updating the time value, enter it in the YYYY:MM:DD HH:mm:ss format.

● Group by values

Click the column value and then select Group by to view grouped results. A new Result tab opensshowing the results that are grouped by the column.

The following illustration displays results grouped by the column sys_bodySize.

23


You can group by different time ranges. Click the timestamp value, and select the Group Dates byoption. From the list, select the option to group your results by different time periods. A new Resulttab opens showing the results that are grouped by different time units.

You can aggregate columns that have Integers and Long values. Click the column value and selectAdd aggregation. Define how to group values in the aggregation column. The options are: SUM,MIN, MAX, AVG. A new column is added in the Data panel.

The following illustration displays a new aggregation column (SUM) added in the Data panel.

24


About ColumnsAll system (event metadata) columns except sys_source and sys_sourceDescription are indexed sosearching is faster on the system columns for many search operations.

The indexes support search operations like CONTAINS, BETWEEN, and all comparison operations,making queries with these filters faster. Operations like LIKE and REGEXP are not supported by theseindexes and are not as fast. The system columns are displayed with the prefix sys_ and all columns frombuilt-in parsers are displayed with the prefix ll_ in the column list.

The following list describes all system columns in the LogLogic Unity event.

Name Type Description

sys_eventTime Timestamp The UTC time of the event in Epochmilliseconds. The sys_eventTime is the timegathered from the event itself.

sys_body String The text of the event.

sys_bodySize Integer The size in number of bytes of the body.

sys_sourceType Integer TIBCO LogLogic® Log ManagementIntelligence (LMI) device type ID.

sys_sourceSubType String This is used to describe more precisely themessage type for a given sys_sourceType.

sys_collectIP InetAddress The IP from where the event originated. Thismust support both IPv4 and IPv6.

sys_collectTime Timestamp The UTC time of the event when it wasingested into the LogLogic Unity event storage.

sys_filename String The file name for event collected from a file.

sys_fileLineNumber Integer The line number in file.

sys_tenant String The customer identifier.

sys_domain String The customer sub-identifier.

sys_partition Long The identifier of the portion of the data on thedata node.

25


Name Type Description

sys_offset Long The location in the LogLogic Unity event store.

sys_eventKey String A unique key that refers to an event in theLogLogic Unity store.

sys_extEventRef String A unique key that refers to an event in theLogLogic LMI event store.

sys_concentratorId String An identifier for the LMI appliance.

sys_sourceDnsName String The DNS name for the event_source_ip.

sys_collectIPZone String The LogLogic LMI collect domain.

sys_source String The name associated with the event sourcedefined by the sys_collectIP+sys_sourceType+sys_collectIPZone triplet. If the collector hasdefined a value for the sys_sourceExtIdcolumn, this value will be used.

sys_sourceDescription String The description of the event source defined bythe sys_collectIP+sys_sourceType+sys_collectIPZone triplet.

DataBased on your search query, the retrieved data is displayed in the normalized tabular format. Eachevent is summarized per row.

You can view data in the following three formats:

● Raw Format● Table Format● Correlation Format

From the Data panel, you can perform the following tasks:

● View event count

The total number of retrieved events is displayed on the upper-right side.

● Filter your results

You can create a filter using the column value and event body text to fine-tune your search results.

Click to show or hide filters from the Data panel.

26


● Add a new source configuration

You can add a new source configuration from the Data panel. Click located on the upper-rightcorner of the Data panel to add a new source configuration. All events that are displayed in theResults tab are copied in the Sources > Source management > Configurations > Add sourceconfiguration > 2. Add sample events and parsing rules > Sample events panel. For instructions onhow to add a new source configuration, see Adding a Source Configuration in Graphical Mode.

If a search query contains a single configuration, then the defined source filter is copied. Ifthere are multiple source configurations defined in the query, the Create source filterpanel does not display any value.

● Edit source configuration

You can edit custom source configurations from the Data panel. Click located on the upper-rightcorner of the Data panel to edit the source configuration. All events that are displayed in the Resultstab are copied in the Sources > Source management > Configurations > Edit source configuration >2. Add sample events and parsing rules > Sample events panel. For instructions on how to updatesource configuration, see Editing Source Configurations.

The button is only visible when search results are retrieved using custom sourceconfigurations. You cannot edit the system source configuration and LogLogic Unity built-in source configurations from the system.

● Download your results

You can share your search results with others. Click located on the upper-right corner of theData panel to download search results in the CSV format.

● Create filtered query as a new search query

After adding filters on your results, click the icon, located on the upper-right corner of the Datapanel, to create a new search query in a new Search tab for the same conditions.

In the following illustration, a filter condition Body INCLUDES vShieldEdge is added on the Datapanel.

Now if you click the icon, a new Search 2 tab is added showing the conditions in the Searchfield.

27


Raw Data FormatBased on your search query, the results are displayed in Raw data format. Each event is summarizedper row. The same result set can be viewed in the Table format.

Using the Raw data format, you can perform the following tasks:

The column value options are displayed in the following illustration.

● Show or hide columns from the Raw data

Click the Columns on or off link to show selected columns below the event or to hide columns toview events in the raw format.

● Wrap long events

Click the Wordwrap text on or off link to indicate if long event should break at normal word breakpoints or to display long events.

● Highlight keywords

28


By default, the Highlight keyword option is set to on for queries that include CONTAINS or LIKEstatements. Click the Highlight on or off link to highlight keywords or remove highlighting fromthe keywords. This option is not visible for queries that do not include CONTAINS or LIKEstatements.

In the following illustration, when the search query is: USE sample | sys_body CONTAINS'Successful', the keyword Successful is highlighted.

● Filter data

Click the column value and select Include this Filter to filter the data based on the value. If youselect Exclude this Filter, the results exclude the specified value.

The Data panel displays results immediately based on the defined filters. You can add multiplefilters to fine-tune your search results. You can update the existing filter value. Click on the value toopen the Enter value field. Update the value in the field and click . The results are refreshedimmediately based on the new filter.

The following illustration displays the Raw data showing filtered results for the sys_bodySize: 65536value.

Click to show or hide filters from the Data panel.

Click the column value and select Include this filter on Result tab to filter the data based on thevalue in a new Result tab. If you select Exclude this filter from Result tab, a new Result tabdisplays results excluding the specified value.

You can filter based on the event body. Drag the mouse to select the event body and select Includethis filter to filter your results based on the event body filter. The selected keyword is highlighted inthe results. If you select Exclude this filter the results exclude the specified event body.

29


● Sort columns

You can sort on any column, including group-by count(*) column, group-by aggregation-columns,and other columns. Click the column value and then select Sort Ascending to sort columns inascending order. Click the column value and then select Sort Descending to sort columns indescending order.

● Group by values

Click the column value and select Group by to view grouped results.

A new Result tab opens showing grouped results for the selected value as shown below.

You can group by different time ranges. Click the timestamp value, then select the Group Dates byoption, and then select the option to group your results by different time periods. The Raw dataview is refreshed showing the results that are grouped by defined time period. When grouped bysys_eventTime, the results are sorted in ascending order.

● Hide columns from the Raw data

Click the column value and then select Hide to hide the selected column from the Raw data format.

Table FormatBased on your search query, the results are displayed in normalized Table format. Each event issummarized per row. The same result set can be viewed in the Raw data format.

Using the Table format, you can perform the following tasks:

● Show or hide event body

30


Click the Messages on or off link to show or hide the event body. Alternatively, hover over the eventnumber link to display the event body.

● Highlight keywords

By default, the Highlight keyword option is set to on for queries that include CONTAINS or LIKEstatements. Click the Highlight on or off link to highlight keywords or remove highlighting fromthe keywords. This option is not visible for queries that do not include CONTAINS or LIKEstatements.

In the following illustration, when the search query is: USE sample | sys_body CONTAINS'Successful', the keyword Successful is highlighted.

● Filter data

Click the column value and then select Include this Filter to filter the data based on the value. If youselect the Exclude this Filter option, the results exclude the specified value.

31


The Table view displays results based on the defined filters immediately. You can add multiplefilters to fine-tune your search results. You can update the existing filter value. Click on the value toopen the Enter value field. Update the value in the field and click . The results are refreshedimmediately based on the new filter.

The following illustration displays the Table showing filtered results for the sys_sourceType: 65536value.

Click to show or hide filters from the Table panel.

Click the column value and then select Include this filter on Result tab to filter the data based onthe value in a new Result tab. If you select Exclude this filter from Result tab, a new Result tabdisplays results excluding the specified value.

You can filter based on the event body. To do this, make sure that the Messages on link is selected.Drag the mouse to select the event body and select Include this filter to filter your results based onthe event body filter. The selected keyword is highlighted in the results. If you select Exclude thisfilter the results exclude the specified event body.

The following illustration shows results based on the event body filter Logon.

32


● Sort columns

You can sort on any column, including group-by count(*) column, group-by aggregation-columns,and other columns. Click the column header and then select Sort Ascending to sort columns inascending order. Click the column value and then select Sort Descending to sort columns indescending order.

● Group by values

Click the column header and then select Group By to view grouped results.

A new Result tab opens showing the grouped results for the selected value as shown below.

You can group by different time range options using the Group Dates by option. Click the timevalue, then select Group Dates by option, and then select the period to group your results bydifferent time periods. The Table panel is refreshed showing the results that are grouped by thedefined time period. When grouped by sys_eventTime, the results are sorted in ascending order.

● Hide columns from the Table

Click the column header and then select Hide to hide the selected column from the Table panel.

33


Correlation FormatThe correlation search results are displayed every time the rule's conditions are met. A correlation Blokis created from a simple correlation rule.

For detailed information on how to define correlation rules, refer to Event Correlation LanguageReference. For detailed information about correlation Bloks, refer to About Correlation Bloks.

1. Type the correlation rule in the Search field. Alternatively, click located next to the Search fieldand select Choose Blok and then select the correlation Blok from the list.

2. Enter the time period in the Time field and click .

The correlation results display all events that contributed to the triggering of the correlation rule.Based on the correlation rule, the columns (correlation events and event groups) are extracted in atable format. Each row helps you analyze the associated values of the columns and event groups.

The following illustration displays the defined correlation rule in the Search field and retrievedevents in the Charts, Columns, and Data panels.

3. Click the event count link to view the event details in a new Search tab.

The event count link is only available when the count is less than 1024.

As shown in the above illustration, click 106 (the event count link), the new search tab opens withthe auto-generated EQL query in the Search field for the events associated with that event count.The Charts, Columns, and Data panels display the results associated for that event count as shownbelow.

34


35


Search Syntax Reference

LogLogic Unity search query language is intuitive and efficient, you can search large data and viewresults in seconds. The search query mainly supports three types of languages: Event Query Language(EQL), Structured Query Language (SQL) dialect, and Event Correlation Language (ECL).

Both EQL and SQL are equally capable for searching, but the syntaxes are different in some cases. Forexample, simply providing a string in EQL is understood as a full text search, but it will give a syntaxerror in SQL. So the translation is not always literal. EQL is easy to use, however, SQL is more familiarand scripting is easy using existing SQL tools.

Using EQL, you can define filters, regular expressions, sources, time ranges. ECL is useful to findpatterns in a given set of data and used for correlation purposes.

Event Query Language ReferenceThe search query supports two types of query languages: Event Query Language (EQL) and LogLogicUnity Structured Query Language (SQL) dialect.

The EQL query is composed of different parts separated by pipe ( | ) character. The pipe delimiter isused to separate the expression and each subsequent expression. Each pipe-delimited expressionfurther processes search results from the preceding expression. For more structured queries, a subset ofSQL is supported that is mainly focused on the SELECT statement. Both languages can be usedinterchangeably; all that is available in EQL can be achieved via SQL and vice versa except thefollowing two differences:

● EQL supports the full text search statement, but SQL does not support this statement. For details, see Filter Statement.

● Multiple EQL filter expression statements, separated by a pipe, get automatically combined usingthe AND operator into a single filter expression. SQL does not support this feature.

The EQL and SQL language rules are based on a Backus-Naur Form (BNF)-like notation as shownbelow:<symbol> ::= <expression> ;

where,

● non-terminal symbols in syntax rules have angle brackets (< >). For example, in the rule<expression> ::= <expression> "+" <integer>; the <expression> is a non-terminal symbol and the rulespecifies that as an expression is the addition of any number of integers.

● terminal symbols are shown in double quotes (" "). For example, the "+" in the previous example.● as an additional shortcut notation to BNF, optional symbols (that can occur zero or one times) are

followed by a question mark (?). For example, in the rule <colNameForSort> ::= <colname> (ASC|DESC)?; a column name used for sort is a column name optionally followed by the keywords ASCor DESC.

● optional symbols that can occur zero or any number of times are followed by an asterisk (*). Forexample, in the rule <itemList> ::= <item> ("," <item>)*; an itemList can contain one or more commaseparated items.

● multiple symbols are grouped together using parenthesis ( ) when some common operation isapplied, for example, the selection of one member of the group, or to indicate that the entire groupcan be repeated zero or more times. An example is shown in the previous bullet item.

● words that are all capitalized represent keywords (special terminal symbols). For example, thekeywords ASC and DESC in the column name for sort described in the previous example.

All parts of the query are optional but the overall the syntax is as shown below:<EQL_statement> ::= <statement> ("|" <statement> )* ;

36


<statement> ::= <useStatement> | <filterStatement> | <groupByStatement> | <columnsStatement> | <sortStatement> | <limitStatement>;

String literals and identifiers (including keyspace, column family names, and source configurationnames) are case-sensitive but all EQL keywords are not case-sensitive. For example, 'USE Windows'and 'use Windows' are treated in the same way.

String literal can be quoted with single (') or double (") quotation marks. The quotation marks (single ordouble) inside the string literal has to be prefixed with backslash ( \ ) character. The \ character changeto be prefixed with another backslash ( \\ ). For example, "Mike's car" or 'Mike\'s car'.

A special syntax for time range can be used. For details, see Time Range Expressions.

In this syntax reference topic, EQL keywords in uppercase letters are used as a convention for easyreadability.

Examples

Expression Definition

sys_sourceType = 65536 and sys_eventTime

in -5d | columns sys_eventTime,

sys_collectIP, ll_eventStatus, ll_type

Events from source type '65536' in last 5 days,display result as a table with columnssys_eventTime, sys_collectIP, ll_eventStatus, andll_type.

use Microsoft_Windows | severity =

'Critical' and user = "Fred" and

sys_eventTime in -1M | group by vm

columns count(*)

Using the source configuration MicrosoftWindows, display results of all critical events fora given user per virtual machine in last month.

Common Search CommandsLogLogic Unity uses the following search commands.

Command Definition

USE Defines the event sources which includes theparsing configuration. For details, see USEStatement.

COLUMNS Defines which columns should appear in searchresults. For details, see COLUMNS Statement.

GROUP BY Groups search results based on specifiedcolumns. For details, see GROUP BY Statement.

SORT BY Sorts search results based on the expression. Fordetails, see SORT BY Statement.

LIMIT Limits the size of search results to be displayed.For details, see LIMIT Statement.

For detailed information about filters, see Filter Statement.

37


USE StatementA source is the name of the log source from which a particular event originates. The source defineswhich events to parse, how to parse them, and what column to extract in order to execute this query.

The USE defines the event log sources which includes the parsing configuration. This is an optionalparameter but it is a good practice to improve performance by reducing the set of event sources and setof parsers used.<useStatement> ::= "USE" <identifier> ( "," <identifier> )* ;

The USE statement consists of the USE keyword followed by one or many source configuration namesseparated by commas. An <identifier> is a letter followed by any sequence of letters, digits, or anunderscore (_).

If you do not specify any source configuration in the Search field, the results are retrieved in this order:first all enabled LogLogic built-in source configurations, next all enabled log sources that are non-LogLogic specific but have source filters defined, and lastly the system log source. The user definedsource configurations without the source filter will not be included in the search query. For a detailedlist of built-in source configurations, see Supported Log Sources. For more information about sourceconfigurations, see Manage Source Configurations.

Certain source configuration expressions refer to a source of infrastructure data. This is defined by thecorresponding event source configuration itself and is typically defined by the name. The currentlydefined infrastructure source configuration records are: Bloks and EventSourceConfiguration thatrepresents the set of currently defined Bloks and Source Configuration records respectively. Forexample, use Unity_config_bloks | COLUMNS name, origin, created, type, description,value

Infrastructure queries may not be mixed with regular search queries. If an infrastructure sourceconfiguration expression is used in a search query, then no event source configuration expression isallowed in the same query and vice-versa. An example of invalid mixed query is: useUnity_config_bloks, system

Examples

Source Configuration Expression Definition

use Windows The result displays all events from Windowssources.

use Windows, Cisco The result displays all events from Windows andCisco log sources.

FILTER StatementA filter is an expression that specifies the conditions that events must satisfy to be returned by thisquery. The filter criteria can be in form of free text search of the entire body or value of a particular pre-parsed or parsed column.

The system (event metadata) columns are indexed so searching is faster on the system columns.

The list of available columns is determined by list of event sources. In case the list of event sources arenot available, the system will do the best to extract those columns using heuristics algorithms. Forqueries, the filter should contain a time condition, otherwise the default is used.

When defining column names in a search query, follow the guidelines described in the COLUMNSStatement section.

38


A filter statement is any expression that evaluates to a result of type boolean. Any event that does notsatisfy this condition will be eliminated from the results. An event that satisfies the condition if itreturns true when the actual event values are substituted for any variable references.

The following table explains the types of filter statements that can be used. For the complete syntax,shown as a BNF grammar, see Filter Syntax.

Operator Description

AND Narrows your search results by only returningthose events where each one of the ANDconditions evaluates to true.

For example, use AND to return resultscontaining all specified keywords. When AND isused, the results contain all specified keywordsand do not contain entries with just one of thespecified keywords.

OR Expands your search results by returning eventswhere either of the OR conditions evaluates totrue.

For example, use OR to return results containingany and all specified keywords. OR is idealwhen you have common synonyms for akeyword. To narrow results as much as possible,combine OR statements with AND statements.

Full text search Full text search on the body of each event can beperformed by simply providing the phrase thatneeds to be enclosed in double quotes. Forexample, use system | "authentication failed"will retrieve all events that contain the abovephrase.

The EQL full text search (specifically onsys_body) is exactly the same as the CONTAINSstatement on the sys_body (so "use system |'Bob'" is exactly the same as "select * from systemwhere sys_body CONTAINS 'Bob'").

Equals (=),

Not equals (<> ), (!=),

Lower than (<),

Lower or equal (<=),

Greater than (>),

Greater or equal (>=)

A comparison condition compares twoexpressions using the operator specified in thecomparison, which may be one of seven possiblecomparison operators with well knownmeanings. The comparison condition evaluatesto true only if the comparison condition issatisfied. This may be used to narrow searchresults. These are case-sensitive.

For example, "col1 > col2/100".

39



Plus (+),

Minus (-),

Multiply (asterisk (*)),

Divide (forward slash (/),

String concatenation (||)

The arithmetic (+,-,*,/) and string concatenation(||) operators can be used to create parts ofother conditions.

For example, "column1 + column2 < 5" or "col3 *4 - 1000 > col5"

The order of evaluation of the operators in anexpression is according to the followingprecedence rules, from highest to lowest, withthe highest precedence implying earlierevaluation:

● Functions

● Multiplication and division: both have equalpriority and the evaluation order is from leftto right

● Addition and subtraction: both have equalpriority and the evaluation order is also fromleft to right

● String concatenation

● Comparators (>, < and so on)

For example, if you have an expression of theform "col1 > col2 + col3*col4", then col3*col4 isevaluated first, and then the result is added tocol2. The col1 is then compared against the finalresult to see if it is greater.

Function A set of predefined functions. For details, see Predefined Functions. They can be used in filter,column expressions, or as part of SourceConfiguration expressions.

The parameters of the functions can beexpressions themselves and will beevaluated before the function is called.

For example, "ToInt(col1 + col2)" will add thecontents of the columns of the event named col1and col2, and pass the result to the ToIntfunction and the result of the function will beused.

BETWEEN Narrows your search results by only selectingthose events where the left hand side expressionevaluates to a value that is between the two righthand side target expressions.

Supports Timestamps, Long, and Integers.

For time range syntax details, see Time RangeExpressions.

40



IN Narrows your search results. This is case-sensitive.

Checks if value matches any one of the values ina set.

For example, "eventID IN ('id1', 'id2', 'id3')".

Supports all data types. For time range syntaxdetails, see Time Range Expressions.

IS NULL, IS NOT NULL Narrows your search results by accepting orrejecting the event based on whether theevaluated expression is null or not null. Anexpression most frequently becomes null if acolumn named in the expression has no valuefor the current event.

Supports all data types.

41



LIKE, NOT LIKE Expands your search results. Returns true if itmatches the supplied pattern. This is case-sensitive. The following rules are used tointerpret the supplied string.

● The character percent (%) is the wildcardcharacter (matches zero or more characters).

● The character underscore (_) means that itmatches exactly one character.

● The backslash character (\) is used to escapeitself and the above two characters if a literalsearch for any is desired.

Since string literals in EQL and SQLrequire backslashes (\) to be escaped,note that additionally escaping for theLIKE statement doubles the escapingrequirement. The simple rule to followis to construct the match string usingthe above rules, then simply doubleup each backslash.

The following examples show the actual syntax(not the escaping needed for Java):

● col1 LIKE "a_b" - produces a match for "acb","adb" and so on

● col1 LIKE "a\\_b" - produces a match for"a_b" but not "acb". Note the doublebackslashes.

● col1 LIKE "a\\\\_b" - produces a match for"a\cb" and "a\db"

● col1 LIKE "a%b" - produces a match for "ab","acb", "accb" and so on

● col1 LIKE "a\\%b" - produces a match for "a%b" but not "acb"

42



CONTAINS, NOT CONTAINS Expands your search results. Returns true whenat least a part of the string matches the suppliedpattern. This is case-insensitive. The sys_bodycolumn is special, because the supplied patternis used to do a full text search on the event body.For all other columns, the following rules areused to interpret the supplied string.

● The character asterisk (*) is the wildcardcharacter (matches zero or more characters).

● The character question mark (?) means that itmatches exactly one character.

● The backslash character (\) is used to escapeitself and the above two characters if a literalsearch for any is desired.

The CONTAINS statement for columns startingwith sys_ uses a full text search.

Since string literals in EQL and SQLrequire backslashes (\) to be escaped,note that additionally escaping for theCONTAINS statement doubles theescaping requirement. The simple ruleto follow is to construct the matchstring using the above rules, thensimply double up each backslash.

The following examples show the actual syntax(not the escaping needed for Java):

● col1 CONTAINS "a?b" - produces a match for"ccc acb jjj", "adb" and so on

● col1 CONTAINS "a\\?b" - produces a matchfor "a?b" but not "acb". Note the doublebackslashes.

● col1 CONTAINS "a\\\\?b" - produces amatch for "a\cb" and "a\db"

● col1 CONTAINS "a*b" - produces a match for"ab", "acb", "accb" and so on

● col1 CONTAINS "a\\*b" - produces a matchfor "a*b" but not "acb"

43



REGEXP, NOT REGEXP Narrows your search results. By default, this iscase-sensitive but can be changed in the regularexpression using the embedded flag (?i).

Returns true if it matches the supplied pattern.The pattern syntax uses POSIX syntax. Sincestring literals in EQL/SQL require backslashes(\) to be escaped, note that all the backslashesinside a regular expression pattern must bedoubled up, similar to the LIKE statement.

Examples:

● col1 REGEXP "[a-z]b" - produces a match for"ab", "cb" but not "Ab" or "_b"

● col1 REGEXP "\\w*" - produces a match fora word, for example "this" or "that", but not"this and that"

Examples

Filter Expression Definition

"Authentication" and sys_eventTime in

-1y

The result displays all events that containAuthentication from the last one year.

use sample | sys_domain = 'samples' |

ll_sourceUser = 'SiteSvrAdmin' |

sys_eventTime in '2014-02-02'

The result displays all events that containcolumn 'll_sourceUser' and value is'SiteSvrAdmin' on the 2 February 2014.

Predefined Functions

The functions that are available in the EQL are listed below.

The smart list functions are usually used in filter expressions and source configuration. The conversionfunctions are typically used when adding a new source configuration, or when you need to define newcolumns, where the expressions for new columns can use conversion functions to convert between datatypes and combine them using various operators. For instructions on how to add a new sourceconfiguration, see Adding a Source Configuration in Graphical Mode.

Function Name Arguments Returns

Smart List functions

lookup (String 1, String 2) The value associated withString2 in the smart list namedString1.

Example: lookup("list1","key1") or $list1("key1")

Conversion functions

44



ToTimestamp (expression, formatString) or(expression, formatString,timezone) or (expression,formatString, timezone,defaultValue)

The expression, which shouldevaluate to a string, isinterpreted as a time accordingto the supplied formatString. Ifthe conversion fails, null isreturned, unless a defaultstring is provided, which isinterpreted as a time andreturned.

Example:ToTimestamp( logFileStringTimestampField, "dd, MM, yyyyHH:mm:ss", "America/Los_Angeles", "01, 01, 197000:00:00")

If timezone isomitted or is empty,the system defaulttimezone is used.

If formatString does notcontain a year, then when thefunction is being evaluated inthe context of processing anevent, the year from the eventtime (sys_eventTime) is used. Ifthis results in a timestamp thatis later than the event time, theprior year will be used.

ToIP (expression) or (expression,defaultValue)

Convert the expression to an IPaddress (Java InetAddress). Ifthe conversion fails, null isreturned, unless a defaultstring is provided, which isinterpreted as an IP addressand returned.

Example: ToIP(ipAddressField,"10.0.0.1")

45



ToTimestampString (expression, formatString) or(expression, formatString,timezone) or (expression,formatString, timezone,defaultValue)

Same as ToTimestamp, exceptit gets converted to string to geta printable timestamp.

Example:ToTimestampString(timestamp, "dd, MM, yyyy HH:mm:ss","America/Los_Angeles", "01,01, 1970 00:00:00")


ToInt (expression) or (expression,defaultValue)

The obvious conversion tointeger with default valuetaken if not convertible.

Example: ToInt("1348") orToInt(numberField, 0)

ToLong (expression) or (expression,defaultValue)

The obvious conversion toLong with default value takenif not convertible.

Example: ToLong("1348") orToLong(numberField, 0)

ToString (expression) or (expression,defaultValue)

The obvious conversion toString with default value takenif not convertible.

Example: ToString(124.5) orToString(numberField, "null")

ToFloat (expression) or (expression,defaultValue)

The obvious conversion toFloat with default value takenif not convertible.

Example: ToFloat("1348.2") orToLong(numberField, 0.0)

LogLogic Unity usesdouble precision(that is 64 bits) whenstoring floating pointnumbers.

ToBool (expression) or (expression,defaultValue)

The obvious conversion toBoolean with default valuetaken if not convertible.

Example: ToBool("FALSE") orToBool(col1, FALSE)

46



ExtractJson (expression, extraction path) or(expression, extraction path,default value)

The expression, which is aJSON string is parsed. A field isextracted from the expressionusing the extraction path. Ifeither the extpression or thepath are invalid, an optionaldefault value is returned.

Example: ExtractJson("{"cat":{"color":" blue"}}", "cat.color","burlesque") would return astring "blue" which is a JSONvalue of color, which is a JSONvalue of cat.

ExtractKvp (expression, extraction path) or(expression, extraction path,nested KVP delimiters /default"{}"/) or (expression, extractionpath, nested KVP, deliiter /default ","/) or (expression,extraction path, nested KVP,delimiter, separator /default"="/) or (expression, extractionpath, nested KVP, delimiter,separator, escape character /default "\\"/) or (expression,extraction path, nested KVP,delimiter, separator, escapecharacter, default value)

The expression, which is anested KVP string is parsed. Afield is extracted from theexpression using the extractionpath. If either the expression orthe path are invalid, anoptional default value isreturned.

Example: ExtractKvp("alert={ agent={ host-name=esbqa01, dns=none}}" ,"alert.agent.dns") would returna string "none".

or

ExtractKvp("(abc^def|asd^aaa)" , "asd", "()", "|", "^")would return "aaa".

String functions

length (expression) Returns the length of the stringvalue of the evaluatedexpression. If the expression isnot a string, for example, aninteger, it will convert it to astring first.

Example: length("abc") is 3,length(3145) is 4 (afterconverting the integer 3145 tothe string "3145")

47



TransformString (stringToTransform,regularExpression, template) or(stringToTransform,regularExpression, template,defaultValue)

It tries to match thestringToTransform with theregular expression, and thenreturns the template withreferences to groups in theregular expression substitutedwith the actual values. To referto groups, use $1, $2, etc torefer to numbered groups, and$<name> to refer to namedgroups. If the string doesn'tmatch, or is there any othererror, the default value will bereturned (or NULL if notspecified).

Example:TransformString("myName=unity" , "myName=(\\S*)", "thename is $1") will return "thename is unity".

lower (String) Lower case of string 1.

upper (String) Upper case of string 1.

trim (String) Trimmed string 1 (withoutleading and trailing spaces).

substitute (String 1, String 2, String 3) Substitute string 2 by string 3in string 1.

left (String, Int) <int> left characters of string 1.

right (String, Int) <int> right characters of string1.

mid

substr

substring

(String, Int 1, Int 2) Characters from string1starting at offset <int1> for alength of <int2>.

find

position

(String 1, String 2) Index of the first occurrence ofstring2 within string1, -1 if nooccurrence is found.

concatenate (String 1, String 2, …) Concatenation of all stringspassed as arguments.

Conditional functions

48



IIF Condition, then, else Returns the Then value ifcondition is true, otherwise itshould return the Else value.

IIF(true, “a”, “b”) returns “a”

IIF(false, ”a”,”b”) returns “b”

Time functions

Seconds Timestamp Truncated the specifiedtimestamp parameter to thesecond.

Minutes Timestamp Truncated the specifiedtimestamp parameter to theminute.

Hours Timestamp Truncated the specifiedtimestamp parameter to thehour.

Days Timestamp Truncated the specifiedtimestamp parameter to theday.

Weeks Timestamp Truncated the specifiedtimestamp parameter to theweek

Months Timestamp Truncated the specifiedtimestamp parameter to themonth.

Years Timestamp Truncated the specifiedtimestamp parameter to theyear.

Time Range Expressions

The time range for IN operator understands both relative time and absolute time. Absolute time is thesame as in BETWEEN operator.

Relative time is defined as: <sign><number><unit>, for example: -5d means 5 days ago.

All dates and times are defined in the local time zone of the machine where the system is installed andit is not based on the browser time zone.

The following time units are available:

● s - second

● m - minute

● h - hour

● d - day

49


● w - week● M - month● q - quarter (3 months)● y - yearThe supported timestamp formats are:

● Any day of the week; for example, MON, TUE, WED, THU, FRI, SAT, SUN● NOW specifies up to the current time● Today specifies as the end of the day (23:59:59)● yyyy-MM-dd HH:mm:ss, {d yyyy-MM-dd HH:mm:ss}, {t yyyy-MM-dd HH:mm:ss}, or {ts yyyy-

MM-dd HH:mm:ss}● MM/dd/yyyy HH:mm:ss● BETWEEN and IN support dates (yyyy-MM-dd or MM/dd/yyyy). The interpretation depends on

whether it is used as beginning or end of time period. When used in beginning it is equivalent toyyyy-MM-dd 00:00:00; when used at the end - yyyy-MM-dd 23:59:59.

Examples

Time Range Expression Definition

-5d Last 5 days including today.

-1M Last month.

"2014-10-20" From 2014-10-20 00:00:00 and 2014-10-2023:59:59.

"2014-10-20":"2014-10-25" From 2014-10-20 00:00:00 until 2014-10-2523:59:59.

"2014-10-20 14:00:00":"2014-10-25

20:00:10"

From 2014-10-20 14:00:00 until 2014-10-2520:00:10.

"2014-10-20 14:00:00":NOW From 2014-10-20 14:00:00 until now (the time thequery was issued).

MON:NOW From beginning of last Monday till the currenttime.

COLUMNS StatementCOLUMNS is used to define which columns should appear in the results and how they should becomputed.<columnsStatement> ::= "COLUMNS" <columnsList> | <aggregationList> ;

<columnList> ::= <columnExpression> ( "," <columnExpression> )* ;

<aggregationList> ::= <aggregationExpression> ( "," <aggregationExpression> )* ;

A COLUMNS statement can be a column based expression or an aggregate expression. A column basedexpression is any expression supporting mathematical and logical operators, functions, and otheroperators. For details, see Filter Statement. An aggregate expression is a similar expression thatcontains an aggregationFunction. If all columns use aggregation functions, the result will containonly one row with results of the aggregation. For details, see GROUP BY Statement.

50


When defining a column name in a search query, it must be enclosed in square brackets ([]) in thefollowing conditions:

● If a column name is also an EQL or SQL keyword, for example, "use MyEventSourceConfiguration |[IN] >5" .

● If a column name has a space, for example, "use Hawk_getProcess | COLUMNS Status, [VirtualKBytes] | sys_eventTime in -10y".

● If a column name contains non-alphabetic or non-digit characters such as dash (-), for example, "[a-b]", to distinguish it from the subtraction expression "a-b".

The following data types for columns are supported:

● String● Integer● Long● Double● Boolean● Timestamp● IP address

Examples

Columns Expression Definition

columns sys_eventTime, ll_collectIP,

sys_body

The result is a table with three columns:sys_eventTime, ll_collectIP, sys_body. Thecolumns could be one of the pre-parsed columnslike sys_eventTime, sys_body, or columns fromconfigured parsers. See USE Statement.

columns count(ll_sourceUser) The result has one column with one row withcount of all events that has ll_sourceUser columnwith no empty value.

columns ToInt(ll_eventActionID)+2 as

action, sys_body

The result is a table with two columns, the firstcolumn called 'action' with the value ofconverting ll_eventActionID to int and adding 2to it, and the second column will be sys_body.

columns max(length(sys_body)) -

min(length(sys_body))

The result is a table with a column containingthe difference in length between the longest andshortest events.

GROUP BY StatementGrouping can be used to group values by one or more expressions involving columns. Groupingrequires a list of grouping expressions and the list of aggregation columns.

A group by expression can be a column name or an expression involving multiple columns, and anoptional list of aggregation functions after the COLUMNS keyword. All the group by expressions andthe aggregates listed after the COLUMNS keyword are displayed by the query.<groupByStatment> ::= "GROUP BY" <columnExpression> ( "," <columnExpression> )* )? (COLUMNS <aggregationFunction> ("," <aggregationFunction> )* )?;

The following aggregation functions are supported:

51


● COUNT(*): Count all the rows.

● COUNT(columnName): Count all the rows in which the value of the column is not null.

● COUNT(DISTINCT columnName): Count all distinct values from the column.

● SUM(column): Sum all values from the column. Supports numerical types (Integer, Long, Double).

● AVG(column): Provide average value for the column. Supports numerical types (Integer, Long,Double).

● MIN(column): Smallest value of the column. Supports all data types that can be ordered (Integer,Long, Double, Timestamp, String).

● MAX(column): Largest value of the column. Supports all data types that can be ordered (Integer,Long, Double, Timestamp, String).

● DURATION(timestamp): Returns the difference (in milliseconds) between the latest and the earliesttime. Supports Timestamp only.

● Time functions: Groups events by time. Supports time functions (Seconds, Minutes, Hours, Days,Weeks, Months, Years).

Examples

Grouping Expression Definition

group by ll_sourceUser columns count(*) The result has two columns, the ll_sourceUserand count of users per distinct value.

group by ll_sourceUser columns

count(ll_sourceUser),

min(sys_eventTime), max(sys_eventTime)

The result has 4 columns ll_sourceUser, numberof users for each distinct value of source user,minimum value of sys_eventTime andmaximum value of sys_eventTime.

group by ll_sourceUser columns

Duration(sys_eventTime)

The result has 2 columns, the source user andthe duration.

group by ToLong(sys_eventTime)/1000,

AVG(sys_bodySize)

The result has three columns,ToLong(sys_eventTime)/1000,AVG(sys_bodySize), and COUNT(*). Groupingis done using the value of the expression in thefirst column, which results in events beinggrouped by the second at which they occurred.The next column shows the average size of theevents every second. The last column shows thenumber of events that occurred every second.

SORT BY StatementSORT BY causes the result rows to be sorted according to the specified expressions. By default, resultsare sorted in ascending order.<sortByStatement> ::= "SORT BY" <expression> ( "," <expression> )* ;

A SORT BY expression can be the name of a column.

If two rows are equal according to the leftmost expression, they are compared according to the nextexpression and so on. If they are equal according to all specified expressions, they are returned in animplementation-dependent order.

The following functions are supported:

52


● ASC: Sort results in ascending order. This is the default order.

● DESC: Sort results in descending order.

Examples

Sorting Expression Definition

sort by sys_eventTime ASC The result is sorted by time in ascending order.

sort by ll_sourceUser, sys_eventTime

DESC

The result is sorted by ll_sourceUser inascending order (default), in case ll_sourceUseris the same, sort by sys_eventTime indescending order.

LIMIT StatementLIMIT indicates the maximum number of results that should be returned by the query.<limitStatement> ::= "LIMIT" <number> ;

If you do not specify a LIMIT clause in the query, the default limit will be used. The default limit is setto 10,000.

Example

Limits Expression Definition

limit 100 Limits the result set to top 100 rows.

Search Examples

SQL Expression EQL Expression Definition

select sys_eventTime,

sys_body from sample

where sys_domain =

'samples' and

sys_eventTime between

'2014-02-02' and

'2014-02-03'

use sample | sys_domain =

'samples' | columns

sys_eventTime, sys_body |


'2014-02-02' and

'2014-02-03'

Displays results from thesample source configurationwhere the records have thetimestamp between '2014-02-02'and '2014-02-03'.

select * from sample

where sys_domain =

'samples' and sys_body

like '%Authentication%'

and sys_eventTime between

'2014-02-02' and

'2014-02-03'


'samples' |

"Authentication" and


'2014-02-02' and

'2014-02-03'

Displays results from thesample source configurationwith "Authentication" in theevent body.

53




where sys_domain =

'samples' and sys_body

like '%logon%' and


'2014-02-02' and

'2014-02-03' limit 10


'samples' | sys_body like

'%logon%' | limit 10 |


'2014-02-02' and

'2014-02-03'

Demonstrates an example of a'like' statement to display alimit of 10 results.


where sys_domain =

'samples' and

sys_eventKey REGEXP '[a-

z0-9|]*' and

sys_eventTime in -10y

limit 10


'samples' | sys_eventKey

REGEXP '[a-z0-9|]*' |

sys_eventTime in -10y |

limit 10

Demonstrates an example ofthe REGEXP expressionmatching.


where sys_domain =

'samples' and


'2014-02-02' and

'2014-02-03' order by

sys_eventTime DESC


'samples' | sys_eventTime

between '2014-02-02' and

'2014-02-03' | sort by

sys_eventTime DESC

Displays events sorted by timefor records having timestampfor the specified dates indescending order.


where sys_domain =

'samples' and


'2014-02-02' and

'2014-02-03' order by

sys_eventTime DESC limit

100




'2014-02-03' | sort by

sys_eventTime DESC | limit

100

Displays top 100 results forrecords sorted by time indescending order.

select sys_eventTime,

sys_body from sample

where sys_domain =

'samples' and


'2014-02-02 14:34:34' and

'2014-02-03 12:00:00'

ORDER BY sys_eventTime

DESC LIMIT 100



between '2014-02-02

14:34:34' and '2014-02-03

12:00:00' | sort by

sys_eventTime DESC | LIMIT

100

Display sorted first page ofresults for events ordered bytime in descending order.

select ll_sourceUser,

count(*) from sample

where sys_domain =

'samples' and


'2014-02-02' and

'2014-02-03' group by

ll_sourceUser


'samples' | group by

ll_sourceUser columns

ll_sourceUser, count(*) |


'2014-02-02' and

'2014-02-03'

Displays grouped results basedon the source users.

54




max(sys_eventTime),

min(sys_eventTime),

count(*) from sample

where sys_domain =

'samples' and


'2014-02-02' and

'2014-02-03' group by

ll_sourceUser


'samples' | group by

ll_sourceUser columns

max(sys_eventTime),

min(sys_eventTime),

count(*) | sys_eventTime


'2014-02-03'

Displays the count of rows fordistinct source users and itscorresponding maximumtimestamp and minimumtimestamp.


(max(ToLong(sys_eventTime

))-

min(ToLong(sys_eventTime)

))/1000 as seconds from

sample where sys_domain =

'samples' and

sys_eventTime IN -10y

group by ll_sourceUser



in -10y | group by

ll_sourceUser COLUMNS

ll_sourceUser,

(max(ToLong(sys_eventTime)

)-

min(ToLong(sys_eventTime))

)/1000 as seconds

Demonstrates the use of acomplex expression in theCOLUMNS statement. For eachuser, calculate the difference intime between the earliest andlatest events. The time valuesare first converted to LONG(milliseconds), then subtracted,and finally divided by 1000 toconvert the milliseconds toseconds.

Event Correlation Language ReferenceLogLogic Unity Event Correlation Language (ECL) is effective in finding patterns in a given set of log.ECL is able to describe searches that are little complex for the regular EQL, especially when there is aneed to join several types of events. Rules described in ECL can be used for advanced forensics searchesand also for real-time alerting.

Rule StructureA rule describes a pattern to look for within a given time window. It contains a list of event groupdefinitions (at least one), and the correlation criteria that are used to join those event groups (if there ismore than one event group). A rule can also be valid for only a given period of time.

All mandatory parameters are explained below. The optional parameters are in square brackets [ ].Valid From yyyy-MM-dd hh:mm:ss To yyyy-MM-dd hh:mm:ss ) ][ <identifier environment> ]USE <source identifier> (, <source identifier>)*Within <integer> [ d |h | m | s ][Fixed | Sliding ]<event group 1><event group 2>…[Correlation<correlation criteria 1><correlation criteria 2>… ][Autofill](Set <expression> AS <identifier>)*[Inject Correlation Event][ LIMIT <integer> CORRELATION EVENTS ]

Each ruleset can have multiple rules. Each rule name must be unique in a defined ruleset.

55


Parameter Description

Rule <identifier environment> The rule name defined using an identifier andthe environment. For details, see IdentifierEnvironment.

USE The list of log sources used by the rule. Multiplelog sources must be separated by comma (,).

Within The time period is defined as an integer in days,hours, minutes, or seconds.

Event Group Each event group describes the criteria that mustcombine events to be group together as part ofthe rule. This is equivalent to a single search inEQL. For details, see Event Group Structure.

Correlation <correlation criteria> The correlation criteria describes the joins andother constraints that various event groups mustmeet to trigger a rule.

For details, see Correlation Criteria.

LIMIT Limit on number of correlation events is onlyeffective for "replay" instances when INJECTCORRELATION EVENT is not set. The default limitis 10,000.

Identifier Environment

An identifier environment is to specify the default tenant or domain or source configuration, whenthose parts are not present in a key identifier.

The identifier environment is composed of:

● [Default Tenant <identifier>]

● [Default Domain <identifier>]

● [Default Source <identifier>]

The identifier environment follows a hierarchical structure when resolving a missing part in anidentifier. The order is as follows:

● Event Group Environment

● Correlation Rule Environment

● Ruleset Environment

● Root Environment (defined outside ECL itself)

— For Correlation REST API: this is the environment parameter.

— For Web application: this is related to the currently logged in user.

Limitations:

● The only possible value for tenant is: tenant1

● The only possible value for domain is: shared

56


● The only possible value for source is: correlation

Simple Identifier

The simple identifier must be defined in letters, numbers, underscore (_), and dollar sign ($) with orwithout single quotes (' '). If not used single quotes (' '), use square brackets ([ ]), or back quotes (` `)

For example: ('a'..'z'|'A'..'Z') ('a'..'z'|'A'..'Z'|'0'..'9'|'_'|'$')*

Key Identifier

An identifier is composed of four parts separated by dots. An identifier part follows the syntax of thesimple identifier. The identifier parts are:

● Tenant name● Domain name● Source configuration name● Field name (or column name)

The key identifier can be defined as shown below:[[[<tenant identifier>.]<domain identifier>.]<source config identifier>.]<field identifier>

The <field identifier> is mandatory part. If the other identifier parts are not defined, they areautomatically extracted from the identifier environment.

Limitations:

● The only possible value for tenant is: tenant1● The only possible value for domain is: shared● The only possible value for source is: correlation

Event Group

An event group describes the criteria events should meet to be part of a rule.

Event groups can be of the following 3 types :

● Required: the rule cannot be triggered if no event(s) matches this event group. This is the defaulttype.

● Excluded: the rule will NOT be triggered if event(s) matches this event group.● Optional: If events are matching this event group, they will be part of the triggering rule if other

criteria are met.

An event group may have the following parts:

● conditions on the number of events● a filtering clause● a grouping clause● a set of having clause● upper limits on the number of groups and events that may be created while this rule is run. This is a

safeguard against a memory overflow.

The event group can be defined as below:Event Group <identifier> [ Is ( Required | Optional | Excluded ) ][ With Delayed Evaluation ]

57


[ At Least <integer> Events ][ At Most <integer> Events ][ <identifier environment> ][ Where <expression> ][ With The Same <expression> [ As <identifier> ]( , <expression> [ As <identifier> ] )* ]( Having <having clause> )*[ Limits <integer> Groups And <integer> Events ]

When At Least parameter is defined, it requires at least an integer more than 0. If it is omitted, thisimplies at least 1.

If Where clause is defined, it should match the expressions. It is evaluated as a Boolean. For details, see Expressions.

Default Limits are 10000 groups and 100000 events.

Expressions

Expressions can be used to express how to compute a value in many situations.

The different situations can be:

● in a condition

● in a grouping definition

● in field assignment

[ ( + | - ) ] <double>[ ( + | - ) ] <long>"<String>"{ ( d | t | ts ) yyyy-MM-dd hh:mm:ss }TrueFalseNull<IPv4 address><IPv6 address><key identifier>$<identifier>(<expression>)( <expression> )<expression> *** <expression><expression> / <expression><expression> % <expression><expression> + <expression><expression> - <expression><expression> Is [ Not ] NullExists <expression><expression> [ Not ] [ Any | All ] Like <expression><expression> [ Not ] [ Any | All ] Contains <expression><expression> [ Not ] [ Any | All ] Regexp <expression><expression> [ Any | All ] = <expression><expression> [ Any | All ] != <expression><expression> [ Any | All ] > <expression><expression> [ Any | All ] >= <expression><expression> [ Any | All ] <= <expression><expression> [ Any | All ] < <expression><expression> [ Any | All ] <> <expression><expression> [ Any | All ] In ( <expression>, expression, … )<expression> In <expression>/<expression><expression> [ Any | All ] Between <expression> And <expression> Case <expression> ( When <expression> Then <expression ) + [ Else <expression> ]<function name> ( [ <expression> ] , [ <expression> ] , … )<aggregation function>

The following operators are supported:

58


● Equals (=)

● Not equals (!=), (<>)

● Lower than (<)

● Lower or equal (<=)

● Greater than (>)

● Greater or equal (>=)

● ~= <expression>: This matches with the regular expression.

● In:

— <list of expressions>: Checks if value matches any one of the values in a set. Supports all datatypes.

— <network>/<net length>: Checks whether an IP address matches a network, defined as anetwork IP address and a network bitmask length.

● Between: Supports Timestamps, Long, and Integers.

● AND

● Functions

● Aggregation Functions

● Identifier Environment

Examples:( sys_eventType = “1234”) and ( sys_body like “%login failed%”)

( sys_bodySize > 30) and (sys_bodySize < 20)

( ll_eventID != null) and ( ll_eventID > -1 )

Functions

Functions are used to compute a value as output from parameters as input.

Some functions are predefined in the language. It is also possible to call a static Java function providedby the user.

Pre-defined Functions

The functions which are available in ECL are listed below.


String functions

len

char_length

character_length

length

(String) Length of string 1.

lower (String) Lower case of string 1.

upper (String) Upper case of string 1.

59



trim (String) Trimmed string 1 (withoutleading and trailing spaces).

substitute (String 1, String 2, String 3) Substitute string 2 by string 3in string 1.

left (String, Int) <int> left characters of string 1.

right (String, Int) <int> right characters of string1 .

mid

substr

substring

(String, Int 1, Int 2) Characters from string1starting at offset <int1> for alength of <int2>.

find

position

(String 1, String 2) Index of the first occurrence ofstring2 within string1, -1 if nooccurrence is found.

concatenate (String 1, String 2, …) Concatenation of all stringspassed as arguments.

TransformString (stringToTransform,regularExpression, template) or(stringToTransform,regularExpression, template,defaultValue)

It tries to match thestringToTransform with theregular expression, and thenreturns the template withreferences to groups in theregular expression substitutedwith the actual values. To referto groups, use $1, $2, etc torefer to numbered groups, and$<name> to refer to namedgroups. If the string doesn'tmatch, or is there any othererror, the default value will bereturned (or NULL if notspecified).

List functions

size List Size of the list.

Conditional functions

IIF Condition, then, else Returns the Then value ifcondition is true, otherwise itshould return the Else value.

IIF(true, “a”, “b”) returns “a”

IIF(false, ”a”,”b”) returns “b”

60



Smart List functions

lookup (String 1, String 2) The value associated withString2 in the smart list namedString1.

isInList (String 1, String 2) True if the value String2 isdefined in smart list namedString1.

Conversion functions

ToTimestamp (expression, formatString) or(expression, formatString,timezone) or (expression,formatString, timezone,defaultValue)

The expression, which shouldevaluate to a string, isinterpreted as a time accordingto the supplied formatString. Ifthe conversion fails, null isreturned, unless a defaultstring is provided, which isinterpreted as a time andreturned.


ToIP (expression_ or (expression,defaultValue)

Convert the expression to an IPaddress (Java InetAddress).Ifthe conversion fails, null isreturned, unless a defaultstring is provided, which isinterpreted as an IP addressand returned.

ToTimestampString (expression, formatString) or(expression, formatString,timezone) or (expression,formatString, timezone,defaultValue)

Same as ToTimestamp, exceptthe conversion is in theopposite direction to get aprintable timestamp.


ToInt (expression) or (expression,defaultValue)

The obvious conversion tointeger with default valuetaken if not convertible.

ToLong (expression) or (expression,defaultValue)

The obvious conversion toLong with default value takenif not convertible.

61



ToString (expression) or (expression,defaultValue)

The obvious conversion toString with default value takenif not convertible.

ToFloat (expression) or (expression,defaultValue)

The obvious conversion toFloat with default value takenif not convertible.

LogLogic Unity usesdouble precision(that is 64 bits) whenstoring floating pointnumbers.

ToBool (expression) or (expression,defaultValue)

The obvious conversion toBoolean with default valuetaken if not convertible.

ToDouble (expression) or (expression,defaultValue)

The obvious conversion toDouble with default valuetaken if not convertible.

ExtractJson (expression, extraction path) or(expression, extraction path,default value)

The expression, which is aJSON string is parsed. A field isextracted from the expressionusing the extraction path. Ifeither the extpression or thepath are invalid, an optionaldefault value is returned.

ExtractKvp (expression, extraction path) or(expression, extraction path,nested KVP delimiters /default"{}"/) or (expression, extractionpath, nested KVP, deliiter /default ","/) or (expression,extraction path, nested KVP,delimiter, separator /default"="/) or (expression, extractionpath, nested KVP, delimiter,separator, escape character /default "\\"/) or (expression,extraction path, nested KVP,delimiter, separator, escapecharacter, default value)

The expression, which is anested KVP string is parsed. Afield is extracted from theexpression using the extractionpath. If either the expression orthe path are invalid, anoptional default value isreturned.

Aggregation Functions

Expressions used in the Having clause must contain at least one aggregation function.Count ( * )Count ( [ Distinct | All ] <expression> Limit <integer> )Sum ( [ Distinct | All ] <expression> Limit <integer> )Avg ( [ Distinct | All ] <expression> Limit <integer> )Max ( [ Distinct | All ] <expression> Limit <integer> )

62


Min ( [ Distinct | All ] <expression> Limit <integer> )Var ( [ Distinct | All ] <expression> Limit <integer> )Stdev ( [ Distinct | All ] <expression> Limit <integer> )

Option Definition

( * ) This applies the function for any event with noadditional constraints.

All This applies the function on all values that arenot null.

Distinct This only applies the function once per distinctvalues.

Sum This is the total value.

Avg This is the average value.

Max This is the maximum value.

Min This is the minimum value.

Var This is the variance.

Stdev This is the standard deviation function.

Having Clause

The Having clause adds additional constraints on the events that have passed the filter and are groupedby the rule.At (Least | Most) <integer> Distinct <expression> As <identifier> Limit <integer>Count Of <expression> Being <expression> (Greater | Less) Than <integer>Percentage Of <expression> Being <expression> (Greater | Less) Than <integer>%<condition>

The Having clause expression must contain at least 1 aggregation function.

The supported parameters are:

● Count Of: count the number of time two expressions are equals and check that this value is greateror less than a boundary.

● Percentage Of: count the number of time two expressions are equals and make a ratio of this countversus the number of events in the group, then check whether the value is below or /above a valueexpressed as percent.

The Having clause can also be an expression using aggregation functions and resolving to a boolean.

Correlation Criteria

Correlation criteria can be of the following three types:

● A join condition describing which fields should be equals in two event groups

● A sequencing constraint that describes the relative order in which two event groups should occurs

63


● An expression criteria that describes a condition among fields of different event groups

<event_group_identifier1>><field_identifier1> == <event_group_identifier2>><field_identifier2><event_group_identifier1> (Begins | Ends) [At Least <integer> [ d | h | m | s ]] [Up To <integer> [ d |h | m | s ]](Before | After) <event_group_identifier2> (Begins|Ends)

This is an expression criteria that is used to describe a condition between fields that belongs to differentevent groups.

<expression which uses syntax eventGroupIdentifier->fieldIdentifier for keys>

For example, group1->sum_bytes >= group2->sum_bytes

The fields referenced in a join must be grouping fields for their respective event groups.

Correlation Blok (ECL) ExamplesWhen a Blok triggers, it creates a correlation event result in forensic or search mode and executes theactions of the associated trigger (create alert, notify by email) in the real-time mode.

Blok Definition

use sample Within 30m Event Group [My Events] where sys_domain = 'samples'

Blok Example 1:

This Blok triggers a new alert atthe first event and willaccumulate all events during 30minutes time period.

use sample Within 30m Event Group [My Events] where sys_domain = 'samples' Having at least 1 distinct [ll_sourceDomain] Having at least 1 distinct [ll_type]

Blok Example 2:

This Blok does the same as BlokExample 1 but the alertsgenerated will then giveinformation about the numberof distinct ll_sourceDomain /ll_type and their values.

use sample Within 30m Event Group [My Events] where [ll_type] ="Network" and sys_domain = 'samples' Having at least 2 distinct [ll_sourceIP]

Blok Example 3:

This Blok filters events whichhave ll_type equal to"Network", and at least 2distinct values of ll_sourceIP.

use sample Within 30m Event Group [suspiciousSources] At least 100 events where [ll_type] ="Network" and sys_domain = 'samples' With the same [ll_sourceIP] Having at least 1 distinct [ll_eventStatus]

Blok Example 4:

This Blok looks for at least 100events with the same criteria asthe previous one, coming fromthe same ll_sourceIP andgiving information about thenumber of distinctll_eventStatus and their value.

64


Blok Definition

use sample Within 30m Event Group [suspiciousUsers] At least 100 events where [ll_type] ="Network" and sys_domain = 'samples' With the same [ll_sourceUser] Having at most 1 distinct [ll_eventStatus] Having at least 10 distinct [ll_sourceIP]

Blok Example 5:

This Blok filters the event thesame way as the previous one,and is looking for 100 eventsfrom the same ll_sourceUserthat have at least 10 distinctll_sourceIP and at most 1distinct ll_eventStatus.

use sample Within 30m Event Group [successAudit] at least 1 events where [ll_type] ="Network" and [ll_eventStatus] = "Success Audit" and sys_domain = 'samples' With the same [ll_sourceUser],[ll_sourceIP] Event Group [failure] at least 1 events where [ll_type] ="Network" and [ll_eventStatus] = "Failure" and sys_domain = 'samples' With the same [ll_sourceUser],[ll_sourceIP] Correlation successAudit->[ll_sourceIP]== failure->[ll_sourceIP] successAudit->[ll_sourceUser]== failure->[ll_sourceUser]

Blok Example 6:

This Blok looks at two groupsof event happening within 30minutes. The first event groupis success audit from the samell_sourceIP/ll_sourceUser andthe second group is failurestatus grouped the same way.

The Blok is triggered if thefields grouped on both eventgroups are same.

use sample Within 30m Event Group [successAudit] is excluded at least 1 events where [ll_type] ="Network" and [ll_eventStatus] = "Success Audit" and sys_domain = 'samples' With the same [ll_sourceUser],[ll_sourceIP] Event Group [failure] at least 1 events where [ll_type] ="Network" and [ll_eventStatus] = "Failure" and sys_domain = 'samples' With the same [ll_sourceUser],[ll_sourceIP] Correlation successAudit->[ll_sourceIP]== failure->[ll_sourceIP] successAudit->[ll_sourceUser]== failure->[ll_sourceUser]

Blok Example 7:

Same as the previous Blok butthis time the Blok is triggered ifthere is only failure eventswithin 30m for the samell_sourceIP / ll_sourceUser.

use sample Within 30m Event Group [users] where [ll_eventStatus] ="Failure" OR [ll_eventStatus] ="Success Audit" and sys_domain = 'samples' With the same [ll_sourceUser] Having at least 2 distinct [ll_eventStatus]

Blok Example 8:

This Blok looks for users thathave events with ll_eventStatusequal to either failure orsuccess.

65


Blok Definition

use sample Within 30m Event Group [users] where [ll_eventStatus] ="Failure" OR [ll_eventStatus] ="Success Audit" and sys_domain = 'samples' With the same [ll_sourceUser] Having at least 2 distinct [ll_eventStatus] Having sum([ll_eventStatus] ="Failure") > ( 2 * sum([ll_eventStatus] ="Success Audit"))

Blok Example 9:

Same as the previous Blok withan additional constraints thatthere are twice as many failuresthan success events.

66


About Bloks

To analyze your data faster, you can create different types of Bloks in LogLogic Unity to help youaccelerate your search process.

A Blok is a contextual element or filter that fits with other elements to form a search query. Build andsave different Bloks that can be used in future searches rather than searching every time with the samefilter. Bloks are reusable elements of a query. You can combine any types of Bloks together, except thecorrelation Blok, to create complex queries.

LogLogic Unity supports the following types of Bloks:

● Filter Bloks: contain filter statements, aggregation rules

● Correlation Bloks: contain different correlation rules

● Time Bloks: contain absolute and relative time ranges

● Source Bloks: contain source statements

You can have one or more filters in a Blok. If you realize that you need to add another Blok to theexisting Blok, you can add more filters and build another Blok. Only one correlation Blok can be used ata time in a query.

You can add new Bloks and modify existing Bloks from the Search tab. Similarly, you can manage all

types of Bloks. On the toolbar, click the Administration icon, the Administration overview landingpage displays different options. Click the Bloks link. For detailed information, see Manage Bloks.

Once the aggregation rule is created, a filter Blok is automatically created for that rule. You cannot editor delete these types of filter Bloks. However, when the aggregation rule is updated or deleted, thecorresponding filter Blok is updated or deleted from the system.

When entering a Blok name in the Search field, start with the prefix defined for each type of Blok aslisted below. Content assist can help you by showing all possible values for that type of Blok.

● time.Blok name

● filter.Blok name

● filter.AGGREGATION_<rule name>

● correlation.Blok name

● source.[sys_concentratorId].Blok name

You can use * instead of sys_concentratorId to select the OR filter of the source statements ofall LogLogic LMI instances that define this source.

For example, create a Blok and use it in a search query:

● Create and save a filter Blok that has user='joe' AND body like '%security%'. Now when you run aquery using this Blok, only events with "joe and security" will be retrieved.

● Use this filter Blok and add another element or filter to it, for example, type user='John' to the samequery to create a more complex query. For example, filter Blok AND user='John'. Now when you runa query using this Blok, events with "joe and security and john" will be retrieved.

Filter BloksYou can create filter Bloks that contain one or multiple filters. Each filter comprises of one or multipleterms. A filter Blok supports valid EQL or SQL statements.

You can have either one or multiple filters in a Blok. If you realize that you need to add another Blok tothe existing Blok, you can add more filters and build another Blok. Multiple Bloks of different types,

67


except the correlation Blok, can be used in a single search query. For detailed information on how tocreate a Blok, see Adding a Blok.

When entering the Blok name in the Search field, start with the prefix filter for any existing filter Blok.Content assist can help you by showing all possible values for that type of Blok. For detailedinformation about valid filters, see Filter Statement.

Correlation BloksFor your forensic needs, search with a created correlation Blok or Event Correlation Language (ECL)rule on historical data. You can create correlation rules to use them for alerts or searches. You canmanage triggers using correlation Bloks so that you can receive alerts on real-time data.

A correlation Blok is a correlation rule without its header (rulename, description, author, and date). Fordetailed information on how to define correlation rules, see Event Correlation Language Reference.

You cannot combine a correlation Blok with other Blok types in a single query. Only one correlationBlok can be used at a time in a query. For detailed information on how to create a Blok, see Adding aBlok.

In a correlation Blok query if there are more than one million events for the defined time duration, onlyfirst one million events are processed by the correlation node for better performance. In such cases, it isa good practice to reduce the time duration to retrieve accurate results.

When entering a Blok name in the Search field, start with the prefix correlation for any existingcorrelation Blok. Content assist can help you by showing all possible values for that type of Blok. Thecorrelation search results are displayed every time the rule's conditions are met. For more information,see Correlation Format.

Viewing All BloksThe default or existing Bloks can be easily used to quickly search your data. The default Bloks havepreset values. You cannot modify or delete the default Bloks. However, you can update or delete anycustom Bloks.

Procedure

1. From the Search page, click located next to the Search field, and select Choose Blok.

2. Select the type of Blok from the list.The options are All, Filter, Correlation, Time, and Source Bloks.

3. In the Find field, type the Blok name to quickly find the desired Blok.

4. Select the Blok name from the list of Bloks.The Description and Source statement fields are auto-populated based on the selected Blok.

5. Click OK to add the Blok in the Search field. If you select a time Blok, it is displayed in the Timefield.

6. Click to view results for the defined Blok.

68


Adding a BlokIf you usually search for events that provide you with specific information such as user name orseverity, you can create a custom Blok for that criteria and save it for later use.

Procedure

1. From the Search page, click located next to the Search field, and select next to the Search field,and select New Blok.

2. Select the Blok type from the list.

3. Enter the name of the Blok in the Name field.It must be a unique name that consists of a single word with no special characters. This is amandatory field.

4. Enter the description of the Blok in the Description field.

5. Enter the statement of the source in the Source statement field.Make sure to enter a valid syntax. Filter and Time Bloks support EQL and SQL syntax. CorrelationBloks support ECL syntax. For syntax information, see Search Syntax Reference.

6. Click Validate to verify the statement. Click Format Statement to format the statement.The Validate option is available only for correlation Bloks.

7. Click Save to save the new Blok.The new Blok is added in the Choose Blok list and displayed in the Search field.

Modifying BloksYou can modify the user-defined custom Bloks at any time. You cannot modify default Bloks. Similarly,you cannot update system generated filter Bloks that have aggregation rule associated. You mustupdate the aggregation rule to update the corresponding filter Blok.

Procedure

1. From the Search page, update the statement in the Search field. Content assist shows you contextualmatches and completions for each keyword as you type into the Search field.For syntax information, see Search Syntax Reference.

2. Click located next to the Search field and select Save as Blok.

3. Update the information.For information about each field, see Adding a Blok.

If you update the ECL rule in a correlation Blok, make sure to deploy the related triggersfor the updated rule to take effect. For details, see Manage Triggers.

4. Click Save to save as a new Blok.The new Blok is added in the Choose Blok list and displayed in the Search field.

Deleting BloksYou can delete the user-defined custom Bloks at any time. You cannot delete default Bloks. Once theBlok is deleted, active queries are not affected, but you cannot start a new query with a deleted Blok.Queries in the Search > History tab that use the selected Blok cannot be started again. When a trigger isactive, you cannot delete the associated correlation Blok. Similarly, you cannot delete system generatedfilter Bloks that have aggregation rule associated. You must delete the aggregation rule to delete thecorresponding filter Blok.

69


Procedure

1. On the toolbar, click the Administration icon, the Administration overview landing pagedisplays different options. To view all Bloks in the system, click the Bloks link.

2. On the Blok management page, select the check box located next to the Blok name that you want to

delete and click .You can select one or multiple Bloks.

3. In the confirmation window, click Ok to delete the selected Blok.The Blok management page is updated immediately.

Time BloksAnalyzing events based on a certain time range can help correlate results and find the root cause faster.You can narrow your search results to a specific time range using the Time Blok. You can use the presettime Blok or create your custom time Blok that you can use any time.

Each time Blok is translated in a statement before executing the query. When entering the time Blokname in the Search field, start with the prefix time for any existing time Blok. You can use ContentAssist to see all possible values for that type of Blok. For detailed information on how to create a timeBlok, see Adding a Time Blok.

By default, the time range is set to last hour. You can define the absolute or relative time. For valid timeranges, see Time Range Expressions.

Viewing All Time BloksThe default or existing time Bloks can be easily used to quickly search your data. The default time Blokshave preset time ranges. You cannot modify or delete the default time Bloks. However, you can updateor delete user-defined time Bloks.

Procedure

1. From the Search page, click located next to the Time field, and select Choose Blok.2. In the Find field, type the Blok name to quickly find the desired time Blok.3. Select the Blok name from the list of Bloks.

The Description and Source statement fields will be auto-populated based on the selected Blok.4. Click Save to add the Blok in the Time field.

The selected time Blok is displayed in the Time field.

5. Click to view results for the defined time range.

Adding a Time BlokIf you usually search for events that are in the specific time range, you can create a custom time Blok forthat time range and save it for later use.

Procedure

1. From the Search page, click located next to the Time field and click Select a date range to opena window.

2. Specify the date and time in the From and To fields. Time must be in Hours and Minutes and clickOK.

70


The selected date and time range is displayed in the Time field.

Alternatively, type in the time expression in the Time field. Content Assist shows you typeahead orcontextual matches and completions for each keyword as you type it into the search field. To definea valid time statement, see Time Range Expressions.

3. To save a new time Blok, click next to the Time field and select Save as Blok. Alternatively, toadd a new Blok, select New Blok.

4. In the Add new Blok window, enter the information in the following fields:a) Name: Enter the name of the Blok.

It must be a unique name that consists of a single word with no special characters. This is amandatory field.

b) Description: Enter the description of the Blok.c) Source Statement: The statement of the source (time expression).

5. Click Save to save the new time Blok.The new time Blok is added in the Choose Blok list.

Modifying Time BloksYou can modify the custom time Bloks at any time. You cannot modify default time Bloks.

Procedure

1. From the Search tab, update the time range expression in the Time field.For detailed information about valid time statements, see Time Range Expressions.

2. To save a new time Blok or update the existing Blok, click next to the Time field and select Saveas Blok.

3. Update the information.For information about each field, see Adding a Time Blok.

4. Click Save to save the new time Blok.The new time Blok is added in the Choose Blok list.

Source BloksYou can narrow your search results to a specific source using the source Blok. All LogLogic LMI devicegroups, that are exported into LogLogic Unity, are reflected as source Bloks in the Blok managementpage. You cannot add, modify, or delete source Bloks.

From the Blok management page, select the Blok name to view the details. The Details panel opens onthe right side of the page. It displays the name, description, type, date the Blok was created, anddynamic rule of the selected Blok. The dynamic rule defines a filter clause that corresponds to thisdevice group.

The Admin > Sources > Source management > Sources page lists all exported LMI log sources, eventhose for which the current user does not have access to the log data. For details, see Manage Sources.

Manage BloksA Blok is a contextual element or filter that fits with other elements to form a search query. Build andsave different Bloks that can be used in future searches rather than searching every time with the samefilter.

For detailed information on how to search using Bloks, see About Bloks.

You can manage all types of Bloks using the Blok management page.

71


On the toolbar, click the Administration icon, the Administration overview landing page displaysdifferent options. To view all Bloks in the system, click the Bloks link. From the Blok managementpage, you can perform the following tasks:

● Find Bloks

You can quickly find the desired Blok by typing the Blok name in the Find field. As you start typingthe Blok name in the Find field, the Blok management page is automatically refreshed showing yourselection.

● View Bloks based on filters

You can use filters to easily find Bloks. Click the View list to view all Bloks in the system.

● Sort Bloks

You can sort any column in ascending or descending order. Click on the column name or click thearrow (that is displayed on the right side of the column name when you click in the column) to sortthe column.

● Add a new Blok

Click to add a new Blok. For instructions, see Adding a Blok.

● Edit existing Bloks

Select the Blok name that you want to update. The Details panel opens on the right side of the page.Click the Edit link to update. For instructions, see Modifying Bloks.

You cannot update system generated filter Bloks that have aggregation rule associated.You must update the aggregation rule to update the corresponding filter Blok.

● Duplicate existing Bloks

Select the Blok name that you want to copy by selecting the check box located next to the Name

column and click to copy the Blok. Enter the new name in the Name field and click OK. You cannow modify the Blok as per your need.

The Duplicate button is enabled after you select a Blok from the list.

● Delete Bloks

You can delete single or multiple Bloks. For instructions, see Deleting Bloks.

You cannot delete system generated filter Bloks that have aggregation rule associated. Youmust delete the aggregation rule to delete the corresponding filter Blok.

● Show or hide columns

You can show or hide columns, except the mandatory column, from the table . Click to view allavailable columns in the table. Select the check box to show the column. Clear the check box to hidethe column from the table. The Blok management page is updated immediately.

The Blok management page information is described below:

72


Column Description

Name The name of the Blok

Description The description of the Blok

Type The type of Blok

Created by The user who created the Blok

73


Manage Dashboard

Dashboard contains a collection of data widgets that provide a graphical representation in the form of achart or count.

The use of dashboard is endless. For example, as an IT administrator, you can focus on all machines inyour enterprise by creating a widget in a dashboard. Dashboards can be built as per your specifications.You can add multiple widgets in a dashboard.

From the Dashboard page, you can perform the following widget tasks:

● Update widget name: Click in the widget name field and update the widget name.

● Refresh widget: Click to refresh the widget. The icon is displayed when you hover over thewidget.

● Configure widget: Click to update the configuration. The icon is displayed when you hover overthe widget.

● View chart details: Hover your mouse over a certain area of the chart to view the details.

● View value details: Hover your mouse over a certain area of the value and click on the value toview details in a Search tab.

● Remove widget: Click to delete widget from the dashboard. The icon is displayed when youhover over the widget.

● Add new widget: Click to add a new widget. For instructions, see Adding Widgets toa Dashboard.

● Mark as a favorite dashboard: Click to mark as a favorite dashboard. A icon indicates thatthe dashboard is marked as favorite.

74


Viewing DashboardYou can view all dashboards, add a new dashboard, copy an existing dashboard, or delete anydashboard in the system.

On the toolbar, click the Dashboard link located on the upper-right corner on the main header. TheDashboard page displays all existing dashboards in the system. From the Dashboard page, you canperform the following tasks:

● Filter dashboards

You can quickly find the desired dashboard by typing the dashboard name in the Find field. As youstart typing a dashboard name in the Find field, the Dashboard page is automatically refreshedshowing your selection.

● View dashboard based on filters

You can use filters to easily find dashboards in the system. Click the View list to view differentfilters.

● Sort dashboards

You can sort the Name column in ascending or descending order on the Dashboard page. Click thecolumn name or click the arrow (that is displayed on the right side of the column name when youclick in that column) to sort the column.


You can show or hide columns, except the mandatory column, from the table . Click to view allavailable columns in the table. Select the check box to show the column. Clear the check box to hidethe column from the table. The Dashboard page is updated immediately.

The Dashboard page information is described below:

Column Description

Name The name of the dashboard.

Click on the icon to mark as favorite. A icon indicates that thedashboard is marked as favorite.

Created by The name of the user who created the dashboard.

Date created The date and time when the dashboard was first created.

Last edited The date and time when the dashboard was last updated.

Adding Widgets to a DashboardYou can create a new dashboard with multiple widgets based on your specifications.

Procedure

1. On the main header, click the Dashboard link to display the Dashboard landing page.

2. From the Dashboard page, click to add a new dashboard.

75


3. To define the dashboard name, click on the Untitled dashboard link to open a field and enter the nameof the dashboard in the field.

4. From the Widget type panel, click on a type of widget that you want to add on the dashboard.The following widget types are available:

● Line: provides results in the form of a line chart

● Bar: provides results in the form of a bar chart

● Pie: provides results in the form of a pie chart

● Number: provides a total count of the results

● Gauge: provides a total count of the results

● Stacked Column: provides comparison in the form of a column chart

● Combined: provides combination results in the form of pie, column, and line chart

5. To define the widget name, click on the Untitled widget link to open a field and enter the name of thewidget in the field.

6. To configure each type of widget, click the Configure link or click . The Settings icon is displayedon the upper-right corner when you hover over the widget panel.The configuration options are different for each type of widget. For more information about eachwidget type, see Line Widget, Bar Widget, Pie Widget, Number Widget, Gauge Widget, StackedColumn Widget, and Combined Widget sections.

7. Click Save to save the widget.The widget is added and the retrieved results are displayed on the dashboard.

Line WidgetThis widget is used to show the distribution of the total count of one selected column over its distinctvalues.

Field Description

Query Enter a search query. Enter USE to start an EQL statement andSELECT to start an SQL statement. You can search based on filterand time Bloks as well.

TimeYou can enter absolute and relative time ranges. Click toopen a window that allows you to define a time range.

For example, enter -5h to display results that occur in the last 5hours.

X-axis Define the column name. If the column names are already definedin the search query, the X-axis column is auto-populated.Otherwise, as you start typing in the field, the available matchingcolumn names are displayed. Choose the column name to definethe X-axis of the line chart.

X-axis label Define the label name for the X-axis that is displayed on the chart.

76


Field Description

Y-axis Define the column name. If the column names are already definedin the search query, the Y-axis column is auto-populated.Otherwise, as you start typing in the field, the available matchingcolumn names are displayed. Choose the column name to definethe Y-axis of the line chart.

Y-axis label Define the label name for the Y-axis that is displayed on the chart.

Categorize by Define the column name by which the Y-axis data will becombined into a series.

Auto refresh Click the slider to ON to refresh the widget. By default, it is set toOFF.

Refresh widget Enter a time interval to refresh the widget. Refresh action startsafter the data is completely retrieved and displayed.

For the search query: use unity | sys_domain = 'internal' | GROUP BY sys_eventTime |COLUMNS sys_eventTime , max(sys_bodySize), avg(sys_bodySize), the X-axis is sys_eventTimeand the Y-axis is max(sys_bodySize), avg(sys_bodySize), an example of a Line widget is shownbelow.

Bar WidgetThis widget is used to show the distribution of the total count of one selected column over its distinctvalues.

Field Description


77


Field Description








Show legends Select the check box to display legends on the chart.

Show inverted Select the check box to invert X-axis and Y-axis values.



For the search query: USE Unity_monitor_memory | sys_domain = 'internal' | COLUMNSsys_eventTime, (memTotal - memFree) as memUsed, memTotal, the X-axis is sys_eventTime, theY-axis shows memUsed and memTotal, an example of a Bar widget is shown below.

78


Pie WidgetThis widget uses one column at a time.

Each pie-slice represents a distinct column value. The Pie widget data varies based on the selectedcolumn. Values that are not displayed in the specified the number of slices, those are grouped togetherinto the Others slice.

Field Description




Slice name Define the column name. If the column name is already definedin the search query, the Slice name column is auto-populated.Otherwise, as you start typing in the field, the available matchingcolumn names are displayed. Choose the column name to definethe slice of the pie.

Slice value The slice value of the pie.

Show up to Enter a number of slices to be displayed on the pie.



For the search query: use sample |GROUP BY ll_eventStatus | COLUMNS ll_eventStatus asEventStatus, count(*) as EventCount, the Slice name is EventStatus and the Slice value isEventCount, an example of a Pie widget is shown below.

79


Number WidgetA numerical value widget displays an important metric for single glance analysis.

Field Description




Show value of Define the column name. As you start typing in the field, theavailable matching column names are displayed. Choose thecolumn name from the list.

Unit Define the appropriate unit. As you start typing in the field, theavailable units are displayed. Choose the appropriate option orenter the desired unit.

Description Enter the widget description this is displayed below the number.

Threshold Define the threshold value. When the number is below thethreshold value, the font color changes to green and when thenumber is above the threshold value, the font color changes tored.



For the search query: use sample | COLUMNS count(*), a threshold value is set to 10000, an exampleof a Number widget is shown below.

80


Gauge WidgetThis widget uses value of a column.

Field Description




Show value of Define the column name. As you start typing in the field, theavailable matching column names are displayed. Choose thecolumn name from the list.

Unit Define the appropriate unit. As you start typing in the field, theavailable units are displayed. Choose the appropriate option orenter the desired unit.

Range Define the range.

Threshold Define the threshold range.



For the search query: use unity | sys_domain='internal' | COLUMNS count(*), the Show valueis count(*) and the Unit is Messages, an example of a Gauge widget is shown below.

81


Stacked Column WidgetThis widget is used to show the distribution of the total count of one selected column over its distinctvalues.

Field Description








82


Field Description




For the search query: use unity | sys_domain='internal' | GROUP BY ll_node, ll_type |COLUMNS ll_node, ll_type, count(*), the X-axis is ll_type, the Y-axis iscount (*), and theCategorize by ll_node is an example of a Stacked Column widget is shown below.

Combined WidgetThis widget is used to show the distribution of the total count of a selected column over its distinctvalues.

Field Description




83


Field Description






Show Average Select the check box if you want to show the average in the lineformat.

Show Total Select the check box if you want to show the total in the pieformat.



For the search query: use unity | sys_domain='internal' | GROUP BY ll_node, ll_type |COLUMNS ll_node, ll_type, count(*), the X-axis is ll_type, the Y-axis is count (*), andCategorize by is ll_node an example of a Combined widget is shown below.

84


Editing a WidgetYou can edit any widget configuration or create a new dashboard with different widgets based on yourspecifications.

Procedure

1. On the main header, click the Dashboard link to display the Dashboard page.

2. From the Dashboard page, click the dashboard name that you want to update.

3. To update a widget, click . The Settings icon is displayed on the upper-right corner when youhover over the widget panel.

4. To change the widget type, click on the icon to display the corresponding configuration fields.The configuration options are different for each type of widget. For more information, see LineWidget, Pie Widget, Number Widget, Gauge Widget, Stacked Column Widget, and CombinedWidget sections.

5. Click Save to save an updated widget on the dashboard.

6. To add a new widget, click the Add widget button located on the upper-right corner of thedashboard.For instructions, see Adding Widgets to a Dashboard.

7. To resize a widget, grab any corner of the widget and resize as per your specifications.

Deleting a WidgetYou can delete any widget from a dashboard at any time.

Procedure


2. From the Dashboard page, click the dashboard name.

3. To delete a widget, click . The Delete icon is displayed when you hover over the widget panel.The dashboard is saved automatically.

Duplicating a DashboardYou can copy the same dashboard as a new dashboard that you allows you to modify as per your need.

Procedure


2. From the Dashboard page, select the dashboard that you want to copy by selecting the check box

located next to the Name column and click to copy the same dashboard.

The Duplicate button is enabled after you select a dashboard from the list.

3. Enter the new name in the Name field and click OK.The newly added dashboard is displayed on the Dashboard page immediately.

85


Deleting a DashboardYou can delete a dashboard at any time.

Procedure


2. From the Dashboard page, select the dashboard that you want to delete by selecting the check box

located next to the Name column and click . To select all dashboards in the system, select thecheck box located next to the Name column header.

The Delete button is enabled after you select one or more dashboards.

3. In the confirmation window, click OK to delete the dashboard and all of its content from the system.The Dashboard page is updated immediately.

86


Administration Overview

An admin (a user with administrator privileges) can manage the LogLogic Unity system and easilymake changes or adjustments to the configurations.

On the toolbar, click the Administration icon, the Administration overview landing page displaysdifferent options:

Click the link in each panel or click the icon from the Administration overview side bar to manage eachfunction:

Icon Function

Expand or collapse the side panel

View the Administration overview page

Manage Bloks

Manage Sources: Configurations and Sources

Manage Rules: Triggers, Forwarding, and Aggregation

Manage Users: Users and Groups

Manage Event storage: Domains and Datanodes

Manage System: Retention policy

87


Icon Function

Manage Collectors

Manage Reports

The LogLogic Unity version number is displayed on the lower-right corner on the Administrationoverview page. Hover over the version number to view the details.

88


Manage Rules

Using the Rules menu, an admin can add, edit, or delete triggers and forwarding rules.

● Triggers: can be activated once a correlation Blok is defined. Triggers describe what action should betaken once a correlation Blok is triggered.

● Forwarding rules: can selectively forward events that are collected into LogLogic Unity to othersystems.

● Aggregation rules: can optimize the performance of aggregation search queries.

Manage TriggersTriggers describe what action should be taken once a correlation Blok is triggered. If there are severaltriggers associated with the same correlation Blok, they all will be triggered.

Triggers can be compressed using the maximum number of triggers per time period setting. You canenable and disable triggers at any time but they must be synchronized in order to be activated on thecorrelation node. An alert is sent out when a trigger is activated in the form of an email or syslognotification. You can set multiple email and syslog notifications for a single alert. For more informationabout alerts, see Monitor Alerts.

You cannot synchronize only one trigger. The synchronization process takes all enabled and disabledtriggers in the system and deploys them to the correlation node.

Viewing TriggersYou can view all defined triggers, add new triggers, edit triggers, enable and disable triggers, anddelete triggers.

On the toolbar, click the Administration icon, the Administration overview landing page displaysdifferent options. From the Rules panel, select the Triggers link. From the Triggers page, you canperform the following tasks:

● Filter triggers

You can quickly find the desired trigger by typing the trigger name in the Find field. As you starttyping a trigger name in the Find field, the Triggers page is automatically refreshed showing yourselection.

● View triggers based on filters

You can use filters to easily find triggers. Click the View list to view different filters.

The Triggers information is described below:

Column Description

Enabled Indicates if the trigger is enabled or disabled.

● ON indicates enabled.

● OFF indicates disabled.

Name The name of the trigger

89


Column Description

Status The status of the trigger. The status options are:

● Active

● Inactive

● Pending

Severity The severity of the trigger. The default options are:

● Info

● Low

● Medium

● High

An admin (a user with administrator privileges) can configureseverity options. The options might differ if they have beenconfigured.

SLA The Service Level Agreement (SLA) time is the time by which an operatoris expected to acknowledge the generated alert.

Description The description of the trigger

Type The type of trigger, an Alert.

Category The category of the trigger. The default options are:

● Attack on third party

● Authorized Activity

● Authorized security testing

● Emergency changes

● False positive

● Known error

● LogLogic Event

● Network Noise

● Security Alert

● Suspicious Activity

● Unauthorized Activity

● Unknown

An admin (a user with administrator privileges) can configurecategory options. The options might differ if they have beenconfigured.

90


Adding a TriggerYou can add a new trigger that can activate an alert. Define the correlation rule associated with the alertand who should receive alert emails and syslog notifications when the trigger is activated. You can alsoset the alert notifications.

Procedure

1. On the toolbar, click the Administration icon to display the Administration overview landingpage.

2. From the Administration overview landing page, click the Triggers link in the Rules panel.

3. From the Triggers page, click to add a new trigger.

4. In the Trigger details section, enter the following information:a) Created by: The name of the creator is auto-populated and cannot be modified.b) Trigger name: The name of the trigger.c) Description: The description of the trigger.d) Severity: Select the severity of the trigger from the list.e) Category: Select the category of the trigger from the list.

An admin (a user with administrator privileges) can configure Severity and Categoryoptions.

5. Select the Correlation Blok that you want to use from the list.You must specify a correlation rule that defines how to gather events into alerts according to alert'svalue fields. For information on how to define correlation rules, see Event Correlation LanguageReference.

6. If you want to add a new correlation Blok, click and select New Blok.For information about how to add a new Blok, refer to Adding a Blok.

7. In the Notifications section, select the type of notification. The options are: Email and Syslog.

● Email

When you select this option, make sure to configure SMTP connection. Forinstructions, see Configuring SMTP Connection.

1. To: The email address of the person who should receive an alert email. Using the comma (,)separator, you can add multiple recipients.

2. CC: The email address of the person who should receive a copy of an alert email. Using thecomma (,) separator, you can add multiple recipients.

3. Subject: The subject of an email.

4. Message: The description of the alert. You can use the defined variables from the list on theright side. Double-click on the variable to add it into the Message field. The variablesmight change based on your data.

● Syslog

1. Host: Enter the destination IP address and the port number. For example,myhostname.mydomain.net:10514.

If you do not enter a port number with the hostname or IP address, the standardsyslog port 514 will be assumed.

2. Protocol: Select the type of protocol. The options are: UDP or TCP. The default is UDP.

91


3. Delimiter: If TCP protocol is selected, define the delimiter. The default value is \n.

4. Facility: Select the option from the list.

5. Severity: Select the option from the list.

6. Message: The description of the alert. You can use the defined variables from the list on theright side. Double-click on the variable to add it into the Message field. The variablesmight change based on your data.

Click to add a new notification. You can add multiple notifications for a single trigger.

8. In the Configure notifications section, enter the following information:a) Enable: Click the slider to ON setting to enable the trigger. Click the slider to OFF setting to

disable the trigger.b) Maximum alerts per time period: Enter the maximum number of alerts to be triggered for aspecified time period. Specify the time period from the list.

9. Click Save to add a new trigger.The newly added trigger is displayed on the Triggers page.

Configuring SMTP Connection

For triggers to send alert notifications, a valid SMTP configuration is required.

Procedure

1. Create a configuration template as shown below.{ "configurations": [ { "smtp": [{ "description": "smtp-1", "hostname": "smtp.gmail.com", "port": 465, "security": "ssl", "username": "", "password": "", "fromAddr": "" }] } ]}

2. Save the file to your local drive.

3. Enter the username and password information.

4. Enter the email address in the fromAddr field.The alert notifications will be sent from this email address.

The fromAddr is a mandatory setting.

5. Run the following command to upload the configuration file:llconf -c <config server host:port> -f <smtp config file path>

Editing TriggersYou can update existing triggers.

Procedure


92



3. From the Triggers page, select the trigger row that you want to update.The Details panel opens on the right side of the page.

4. Click the appropriate Edit link to update that section.For information about each section, see Adding a Trigger.

5. Click Save to save the updated information.The updated trigger is displayed on the Triggers page.

What to do next

Whenever you update any trigger in the system, you must synchronize and deploy all triggers.

Synchronizing TriggersThe synchronization process takes all enabled and disabled triggers and deploys them to the correlationnode. This process resets all triggers in the system.

Procedure



3. From the Triggers page, update the trigger information. For details, see Editing Trigger.

When the triggers are updated, the number of updates are displayed on the Sync button.

The Sync button is enabled only when there are any updates to the existing triggers.

4. Click to reset all triggers.A confirmation window is displayed showing all triggers in the system that will be reset.

5. Click Sync to reset all triggers and deploy them to the correlation node.

Once all triggers are reset, the Sync button is disabled on the Triggers page.

Enabling or Disabling TriggersTriggers can be enabled or disabled and must be synchronized in order to be activated on thecorrelation node.

Procedure



3. To enable the trigger, click the slider in the Enable column to ON.

4. To disable the trigger, click the slider in the Enable column to OFF.

What to do next

Once you update the trigger, click the Sync button to reset and deploy all triggers.

93


Deleting TriggersYou can only delete disabled triggers.

Procedure



3. From the Triggers page, select the trigger that you want to delete by selecting the check box located

next to the Enable column and click .

The Delete button is available only for disabled triggers.

If the trigger is enabled, click the slider to OFF to disable the trigger and then you can delete thetrigger.

4. In the confirmation window, click Delete to delete the trigger from the system.The Triggers page is updated immediately.

Manage Forwarding RulesYou can define rules that can selectively forward all matched events that are collected into LogLogicUnity to other systems. Based on real-time search query, all matched events are forwarded to thedefined destination system. When the new events are collected into the LogLogic Unity system, thoseevents that match the query criteria are formatted as per the rule before forwarding to the destinationsystem.

Viewing Forwarding RulesAn admin can view all forwarding rules, add a new, edit, or delete rules from the system.

On the toolbar, click the Administration icon, the Administration overview landing page displaysdifferent options. From the Rules panel, select the Forwarding link. From the Forwarding page, youcan perform the following tasks:

● Filter rules

You can quickly find the desired rule by typing the rule name in the Find field. As you start typing arule name in the Find field, the Forwarding page is automatically refreshed showing your selection.

● View rules based on filters

You can use filters to easily find rules in the system. Click the View list to view different filters.

● Sort rules

You can sort any column in ascending or descending order on the Forwarding page. Click thecolumn name or click the arrow (that is displayed on the right side of the column name when youclick in that column) to sort the column.


You can show or hide columns, except the mandatory column, from the table . Click to view allavailable columns in the table. Select the check box to show the column. Clear the check box to hidethe column from the table. The Forwarding page is updated immediately.

The Forwarding page information is described below:

94


Column Description

Enabled The status of the rule. Click the slider to change the rule status.

● ON: Enabled

● OFF: Disabled

Name The name of the rule.

Description The description of the rule.

Query The defined search query for the rule.

Type The type of the rule.

Adding a Forwarding RuleBased on real-time search query, all matched events are forwarded from the LogLogic Unity to thedefined destination system. Make sure you enable the forwarding rule to start sending events to thedestination system.

Procedure


2. From the Administration overview landing page, click the Forwarding link in the Rules panel.

3. From the Forwarding page, click to add a new forwarding rule.

4. Enter the rule name in the Name field.

5. Enter the description in the Description field.

6. Enter the search query in the Query field.Make sure to enter a valid syntax of a search query. These queries do not support aggregation,group by, or sorting functions. Click Validate to verify the search statement. Click Format to formatthe statement.

7. To activate the forwarding rule, click the Active slider to ON.The forwarding rule must be activated so that events can be forwarded from LogLogic Unity to thedestination system.

8. From the Type list, select Syslog.

9. Enter the destination IP address and the port number of the syslog server that connects withLogLogic Unity in the Host field. This is a mandatory field.

For example, myhostname.mydomain.net:10514. If you do not enter a port number with thehostname or IP address, the standard syslog port 514 will be assumed.

10. Select the type of protocol from the Protocol list. The options are: UDP and TCP.

11. By default, the Add syslog header option is set to ON. To disable the option, click the Add syslogheader slider to OFF.

12. If the Add syslog header option is enabled, define the following options:a) Facility Column: Select the appropriate column name based on the query from the list. The

columns that contain the INT and LONG data types are displayed. If you choose ConstantFacility, select the value from the list.

95


b) Severity Column: Select the appropriate column name based on the query from the list. Thecolumns that contain the INT and LONG data types are displayed. If you choose ConstantSeverity, select the value from the list.

c) Timestamp Column: Select the column name from the list. The columns that contain the datatype TIMESTAMP are displayed. By default, the sys_collectTime is set.

d) Application Column: Select the column name from the list. The columns that contain the datatype STRING are displayed. If you choose Constant Application Value, enter a value in theConstant Application Value field. By default, unity is set.

e) Source Host Column: Select the column name from the list. The columns that contain the typeSTRING and INET_ADDR are displayed. If you choose Unity IP Address, the host address of themachine where LogLogic Unity is running is set.

13. In the Message template field, enter the description of the syslog message. It can contain a textmessage and references to column names for the rule's search query. After you validate the searchquery, the right-side panel displays all column names that are associated with the search query.Double-click on the variable to add it into the Message template field. The variables might changebased on your specified search query.For example, if you define a message as: "The user {ll_sourceUser} logged in". Then, for each eventthat matches the rule's query, the rule generates a syslog message with the defined text, but thevariable {ll_sourceUser} is replaced with the value of the ll_sourceUser column in that particularevent.

If the Add syslog header option is disabled, make sure to enter the full syslog format in theMessage template field before selecting the variables to add to the message. For example, <13>2015-01-01T12:12:12 localhost unity: Listening at address {sys_collectIP}

14. Click Save to add a new forwarding rule.The newly added rule is displayed on the Forwarding page.

Editing a Forwarding RuleYou can update any forwarding rule at any time. When the existing rule is updated in the LogLogicUnity system, all matched events for the new rule are forwarded to the destination system.

Procedure



3. From the Forwarding page, select the rule name that you want to update.The Details panel opens on the right side of the page.

4. Click the Edit link to update the rule information. Make the necessary updates.For more information about fields, see Adding a Forwarding Rule.

5. Click Save to save the updated information.The updated rule is displayed on the Forwarding rules page.

Enabling or Disabling Forwarding RulesForwarding rules can be enabled or disabled in order to be activated or deactivated. Once the rule isactivated, all matched events are forwarded to the destination system.

Procedure


96



3. To enable a rule, click the slider in the Enabled column to ON.

4. To disable a rule, click the slider in the Enabled column to OFF.

Deleting a Forwarding RuleYou can delete any forwarding rule any time. Once the rule is deleted from the LogLogic Unity system,forwarding will stop immediately.

Procedure



3. From the Forwarding page, select the forwarding rule that you want to delete by selecting the check

box located next to the Enabled column and click . To select all rules in the system, select thecheck box located next to the Enabled column header.

The Delete button is enabled after you select one or more rules.

4. In the confirmation window, click Ok to delete the rule.The Forwarding page is updated immediately.

Manage Aggregation RulesYou can define aggregation rules to optimize the performance of aggregation queries. Once created andenabled, the system pre-computes the aggregations as LogLogic Unity events arrive in the system. Astime progresses, the pre-computed aggregates accumulate providing results much faster than thequeries that were not optimized. Such optimized aggregation queries can be vital for creatingresponsive dashboards.

An aggregation rule is defined as a regular EQL or SQL query that contains a GROUP BY statementand aggregated projections. For details, see GROUP BY Statement.

For example, use Hawk_getProcess | GROUP BY Domain, ProcessName, AgentName COLUMNSavg(Memory), sum(PageFaults), max(NoOfThreads)

For the above search query, the aggregation rule maintains the aggregate metrics avg(Memory),sum(PageFaults), max(NoOfThreads) for each combination of the GROUP BY fields that are Domain,ProcessName, and AgentName. The computation happens on every event created in the system.

When you use this query on the Search page, or a query that uses a subset of this data, the query isrouted to the collected aggregated data of the system instead of non-aggregated data. The aggregateddata already has pre-computed metrics and returns the results without any computation.

Such optimized aggregation queries are very sensitive to the time filter conditions. The query matchingan aggregation rule must have time filter condition greater than the rule creation time and the ruleretention period. For example, if the rule was created on 10 am , Jan 1, 2016 with the retention period of2 weeks. After 2 days, on Jan 3rd if you query for aggregated data which matches the rule but withfilter condition of -2w, the result should still be un-optimized because 2 weeks from the time of searchwill result in time range starting from Dec 19th, and the aggregation rule was not created on Dec 19th.

Aggregation rules also support grouping by time aggregates. The supported time aggregations are:

● Group by Years: using the scalar function years(sys_eventTime)

97


● Group by Months of the year: using the scalar function months(sys_eventTime)

● Group by Weeks of the month: using the scalar function weeks(sys_eventTime)

● Group by Days of the week: using the scalar function days(sys_eventTime)

● Group by Hours of the day: using the scalar function hours(sys_eventTime)

● Group by Minutes of the hours: using the scalar function minutes(sys_eventTime)

For example, use Hawk_getProcess | GROUP BY Domain, ProcessName, AgentName,days(sys_eventTime), hours(sys_eventTime), minutes(sys_eventTime) COLUMNS

avg(Memory), sum(PageFaults), max(NoOfThreads)

This rule computes the metrics avg(Memory), sum(PageFaults), max(NoOfThreads) for eachcombination of Domain, ProcessName, AgentName across each possible minute of the hour, each hour,and each day.

Such queries retrieve the time series data for trend analysis. For example, min(Memory), max(Memory),avg(memory) for Agent='Agent1' AND Process='Process1' AND Domain='domain1' aggregated for eachhour for the day='Monday'".

This query can be used in a dashboard to create a time series chart showing the trend, for example,average memory usage for a process across hours of the day.

An aggregation query can be equal or subset of another rule query if the FROM clause are equal,GROUP BY non-time aggregates time are exactly equal and time aggregates are equal or subset and theprojection aggregates of the two queries are equal or subset.

Examples:

● Rule1 query is: use system | GROUP BY sys_tenant, sys_domain, weeks(sys_eventTime),days(sys_eventTime), hours(sys_eventTime) COLUMNS max(sys_bodySize), avg(sys_offset)

If another Rule2 query has the FROM clause as (use system), then Rule2 FROM clause is equal tothat of Rule1.

● Rule1 contains the GROUP BY clause as: 'sys_tenant, sys_domain, weeks(sys_eventTime),days(sys_eventTime), hours(sys_eventTime)' and Rule2 contains the GROUP BY clause as: tenant,domain, days(sys_eventTime) , then Rule2 GROUP BY clause is a subset of Rule1 GROUP BYclause.

● Rule1 contains the projection aggregates as: max(sys_bodySize), avg(sys_offset) and Rule2 containsthe projection aggregates as: avg(sys_offset), then Rule2 Projection aggregates are subset of Rule1.

Viewing Aggregation RulesYou can view all aggregation rules, add a new rule, edit existing rules, or delete rules from the system.

On the toolbar, click the Administration icon, the Administration overview landing page displaysdifferent options. From the Rules panel, select the Aggregation link. From the Aggregation page, youcan perform the following tasks:

● Filter rules

You can quickly find the desired rule by typing the rule name in the Find field. As you start typing arule name in the Find field, the Aggregation page is automatically refreshed showing your selection.

● View rules based on filters

You can use filters to easily find rules in the system. Click the View list to view different filters.

● Sort rules

98


You can sort any column in ascending or descending order on the Aggregation page. Click thecolumn name or click the arrow (that is displayed on the right side of the column name when youclick in that column) to sort the column.


You can show or hide columns, except the mandatory column, from the table . Click to view allavailable columns in the table. Select the check box to show the column. Clear the check box to hidethe column from the table. The Aggregation page is updated immediately.

The Aggregation page information is described below:

Column Description

Enabled The status of the rule. Click the slider to change the rule status.

● ON: Enabled

● OFF: Disabled

Name The name of the rule.

Aggregation start date The time when the optimization started.

Description The description of the rule.

Query The aggregation query for the rule.

Adding an Aggregation RuleWhen a new aggregation rule is created, it starts computing the metrics for the given GROUP BYexpressions only for new events arriving in to the system. It does not aggregate data for the eventsgenerated before the rule creation time. When an aggregation query matches an aggregation rule, thequery results are fetched from the collected aggregation data. Once the aggregation rule is created, afilter Blok is created automatically for that rule that can be used in future searches.

Procedure


2. From the Administration overview landing page, click the Aggregation link in the Rules panel.

3. From the Aggregation page, click to add a new aggregation rule.

4. In the Create by field, the user name who created the aggregation rule is displayed automatically.

5. Enter the rule name in the Name field.This field name cannot be blank, cannot contain special characters, and duplicate names are notallowed.

6. Enter the description of the rule in the Description field. This is an optional field.

7. Enter the search query in the Query field.Make sure to enter a valid syntax of a search query. A regular EQL or SQL query that containsGROUP BY statement and aggregated projections are supported. For details, see GROUP BYStatement. Note the following exceptions:

99


● GROUP BY statement should not be any scalar expression other than the time function, forexample, hours(sys_eventTime) is supported but 'length(sys_domain)/2' is not supported.

● Time functions in the GROUP BY statement can only have the sys_eventTime as an argument.Any other timestamp column is not supported.

Click Validate to verify the search statement.

8. Select the appropriate Aggregation time check box to add time functions in the query. Clear thecheck box to remove the time functions from the GROUP BY clause of the query.When you select this option, the rule query is modified to insert the defined time aggregation in theGROUP BY clause. These default time aggregation functions that will be inserted:weeks(sys_eventTime), days(sys_eventTime), hours(sys_eventTime), minutes(sys_eventTime).

9. To activate the aggregation rule, click the Enabled slider to ON.The optimization starts after the rule is enabled. The disabled rule stops computing real-timeevents.

10. Enter the retention time in the Retention period field for which the computed aggregation valuesremain in the aggregated data so that you can search based on the same aggregation functions untilthe specified time has passed. By default, it is set to -1w (1 week).

For example, if the retention period is -2w (2 weeks), then a computed metric for a dimensionremains in the system for at least 2 weeks since the time of creation.

11. Click Save to add the new aggregation rule.The newly added rule is displayed on the Aggregation page.

Editing an Aggregation RuleYou can update any aggregation rule at any time. When the existing rule is updated in the LogLogicUnity system, the aggregated data is reset and computation starts again from that time. When theaggregation rule is updated, the corresponding filter Blok is also updated automatically.

Procedure



3. From the Aggregation page, select the rule name that you want to update.The Details panel opens on the right side of the page.

4. Click the Edit link to update the rule information. Make the necessary updates.For more information about fields, see Adding an Aggregation Rule.

5. Click Save to save the updated information.The updated rule is displayed on the Aggregation page.

Deleting an Aggregation RuleYou can delete any aggregation rule at any time. Once the rule is deleted from the LogLogic Unitysystem, the aggregated data will be deleted for that rule and the query will not be optimized. Similarly,the corresponding filter Blok is also deleted from the system.

Procedure



100


3. From the Aggregation page, select the aggregation rule that you want to delete by selecting the

check box located next to the Enabled column and click . To select all rules in the system, selectthe check box located next to the Enabled column header.

The Delete button is enabled after you select one or more rules.

4. In the confirmation window, click Ok to delete the rule.The Aggregation page is updated immediately.

Enabling or Disabling Aggregation RulesAggregation rules can be enabled or disabled in order to be activated or deactivated. The optimizationstarts after the rule is enabled. When the rule is disabled, it stops computing aggregations.

Procedure



3. To enable a rule, click the slider in the Enabled column to ON.

4. To disable a rule, click the slider in the Enabled column to OFF.

101


Monitor Alerts

An alert is generated when real-time events are matching a correlation Blok that has an active triggerlinked to it. An alert can also be distributed by an email to a pre-defined list of people.

By default, the retention period is 90 days for all generated alerts. For email notifications, ensure thatyou configure SMTP connection. For instructions, see Configuring SMTP Connection.

For information on how to define triggers, see Manage Triggers.

Viewing AlertsYou can view all triggered alerts, acknowledged alerts, and filter on the existing alerts in the system.

From the Alerts page, you can perform the following tasks:

● Filter alerts

You can quickly find the desired alert by typing the alert name in the Find field. As you start typingthe alert name in the Filter field, the Alerts page is automatically refreshed showing your selection.

● View alerts based on filters

You can use filters to easily find alerts. Click the View list to view different filters.

● All - view all alerts in the system.

● Custom - view based on user-defined filters. For example, High Severity, Acknowledged, andUnacknowledged.

● Acknowledge alerts

Acknowledging an alert indicates that you have recognized the alert. Once you acknowledge thealert, your user name gets associated with that alert. For instructions on how to acknowledge alerts,see Acknowledging Alerts.

● Auto-refresh Alerts table

Click the down arrow next to the refresh button to set the refresh interval in seconds. Enter thetime in seconds. The Alerts table is refreshed as per the defined time interval. By default, it isrefreshed every 30 seconds.

● Sort alerts

You can sort any column in ascending or descending order. Click on the column name or click thearrow (that is displayed on the right side of the column name when you click in that column) to sortthe column.


You can show or hide columns, except the mandatory column, from the table . Click to view allavailable columns in the table. Select the check box to show the column. Clear the check box to hidethe column from the table. The Alerts page is updated immediately.

The Alerts information is described below:

102


Column Description

Severity The severity of the trigger. The default options are:

● Info

● Low

● Medium

● High

An admin (a user with administrator privileges) can configureseverity options. The options might differ if they have beenconfigured.

SLA Expiration The Service Level Agreement (SLA) expiration time is the time by whichan operator is expected to acknowledge the alert. When the SLA timeexpires, it displays the time in negative hours or days in the field.

Status The icon indicates if the alert is active or expired.

● The indicates that the alert is expired.

● The indicates that the alert is acknowledged.

AcknowledgedThe indicates that the alert is acknowledged.

Name The trigger name associated with the trigger.

Description The description of the alert.

Category The category of the trigger. The default options are:

● Attack on third party

● Authorized Activity

● Authorized security testing

● Emergency changes

● False positive

● Known error

● LogLogic Event

● Network Noise

● Security Alert

● Suspicious Activity

● Unauthorized Activity

● Unknown

An admin (a user with administrator privileges) can configurecategory options. The options might differ if they have beenconfigured.

103


Column Description

Notified The date and time of the alert notification.

Elapsed time The time since the alert was created.

Last updated The time when the alert is last updated.

Acknowledging AlertsAcknowledging an alert indicates that you have received and recognized the alert. Once youacknowledge the alert, your name gets associated with that alert.

Procedure

1. From the Alerts page for a single or multiple alerts, select the check box located on the left side ofthe alert. For all alerts in the system, select the check box located on the left side on the top of thetable.

2. Click Acknowledge to acknowledge selected alerts.

3. In the Acknowledge Alerts window, the name of a person who acknowledges the alert is auto-generated in the By field.

4. Select the alert Severity from the list.

5. Select the alert Category from the list.

6. Enter any comments in the Comment field.

7. Click Acknowledge to acknowledge alerts.

The Alerts table shows icon in the Status column and a checkmark in the Acknowledgedcolumn for all acknowledged alerts.

Viewing Alert DetailsYou can view details of any generated alert.

Procedure

1. From the Alerts page, click the alert name to view the alert details.In the Details window, you can view alert details, history, associated correlation rule, and eventgroup details.

104


2. To acknowledge the alert, click Acknowledge. For details, see Acknowledging Alerts.

Viewing Event Group DetailsEach event group describes the criteria that must combine events to be grouped together as part of thecorrelation rule. This is equivalent to a single search query defined in EQL.

Procedure

1. From the Alerts tab, click the alert Name to view the alert details.In the Details window, you can view alert details, history, associated correlation rule, and eventgroup details.

2. Click the event group count link (as shown in the above illustration) to view the associated eventcount query.A new search tab is added showing the event count query in the Search field. The Result tabdisplays the retrieved results in the Charts, Columns, and Data panels.

The following illustration displays a new Search tab opened with the event count query added inthe Search field and retrieved results in the Charts, Columns, and Data panels.

105


106


Manage Source Configurations

LogLogic Unity parses log data into a structured format to enhance search and analysis. Based on thelog source type, you can define how to parse your data and which columns to extract.

Source configuration helps you define parsing rules that extract columns from your data. If the sourceconfiguration has more than one parsing rule defined, then the extracted column set is a union of allparsing rules and additional system defined columns. For example, create a source configuration anddefine parsing rule, Rule1 to extract four defined columns and Rule2 to extract eight different definedcolumns. Now, when you run a search query on this source configuration, the 12 columns are extractedfrom your data and values are displayed as defined by the corresponding parsing rule.

Parsing rules are applied in the order they are defined in a source configuration. For example, if Rule1matches some of your data then it will be used to extract column values. If Rule1 fails to match withyour data, then only Rule2 is applied, and so on. You can change the order of parsing rules.

You can add source configurations using two different modes:

● Graphical mode: This is a default mode. A wizard helps you add source configuration and theassociated rules. For details, see Adding a Source Configuration in Graphical Mode.

● Raw mode: This is for advanced users who understand the JSON syntax. Use JSON syntax to add asource configuration and associated rules. For details, see Adding a Source Configuration in RawMode.

You can switch between both modes at any time. All information associated with that sourceconfiguration is preserved and is available when you switch from one mode to another.

You can create a source configuration that defines which log source to use for parsing based on the datarelevance. For multiple log sources, the order of precedence can be defined in a specified query. Thesystem columns are extracted from event metadata. All system columns are displayed with the prefixsys_ and all columns from built-in parsers are displayed with the prefix ll_ in the Columns panel.

LogLogic Unity provides built-in source configurations. For a detailed list, see Supported Log Sources.

LogLogic Unity supports the following types of parsers:

● Key-value Parser: This parser uses simple key-value pair parsing rules to extract keys and values.The parser recognizes patterns like k1=v1, k2=v2, k3=v3. You can use key-value pair separators, forexample, space, comma (,), or semi-colon (;), and key and value separators, for example, equal sign(=) or colon (:). Separators can be either one or more characters that have to be matched exactly orthey can be regular expressions.

When referring to a value in a column expression, it is referred to as $<key name>. So for a key withname ‘user’ the value is referred to as $user.

Regular expressions can also be used to parse data from the beginning and ending of the event. Thiscan be useful when parsing events that either start with or end with data that is not in the key-valuepair format. If these regular expressions contain named groups, then those groups are extracted andcan be used to populate columns.

It is also possible to specify the name of the last key in the data. Any data after that last key istreated as the value of that last key. This can be useful in situations where the last value in the datacontains characters that might be interpreted as separators.

● Columnar Parser: The data is extracted into different columns. This parser operates on data that isseparated by a character or a sequence of characters, for example, comma, or tab. There is no key-value, just the value. The data from different log sources extract different columns depending onkeys identified in the data. When referring to a column in a column expression, it is referred to as$<column number>. So the first column is referred to as $1, the second column is $2 and so on.

107


● Regex Parser: Regular expressions (Regex) are a sequence of characters that form a search pattern,mainly for use in pattern matching with strings or string matching. LogLogic Unity can use regularexpressions for extracting columns from matched events.

A working knowledge of regular expressions is a prerequisite.

Each character in a regular expression is either a meta character with its special meaning, or aregular character with its literal meaning. Together, they can be used to identify textual material of agiven pattern, or process a number of instances of it that can vary from a precise equality to a verygeneral similarity of the pattern.

LogLogic Unity supports the regular expression meta characters, based on Java regular expressions.For details, see Supported Regular Expression Characters.

Columns are extracted using either the capturing group pattern (simple parenthesis), the namedcapturing group pattern (?<name>), or a combination of both. When referring to a column in acolumn expression, when using named capturing groups the column name will be that specified bythe group name, preceded by “$”. When using unnamed capturing groups, the name will be “$”followed by the group index. So the first unnamed group column is referred as $1, the second as $2,and so on, while a group named “user” is referred as $user. When using a combination of namedand unnamed capturing groups, the named capturing group columns must be referred to by theirgiven names rather than by "$" followed by their index.

● CEF Parser: HP ArcSight Common Event Format (CEF) is an open log management standard. CEFdefines a syntax that comprised of a standard header and a variable extension, formatted as key-value pairs. Based on the ArcSight Extension Dictionary, the CEF header columns Version, DeviceVendor, Device Product, Device Version, Signature ID, Name, and Severity are extracted intocolumns with their names, and expressions set to $cefVersion, $cefDeviceVendor,$cefDeviceProduct, $cefDeviceVersion, $cefSignatureID, $cefName, and $cefSeverity respectively.

The name of a column for an extension listed in the ArcSight Extension Dictionary is the full name ofthe extension. The name of a column for an extension that is not listed in the ArcSight ExtensionDictionary is the key name as it is displayed in the data preceded with “$”.

The expressions of the non-timestamp extension columns are the CEF Key Names as defined in theArcSight Extension Dictionary. The expressions of the timestamp extension columns are of the formToTimestamp(<$CEF Key Name>, <proposed format>) where <proposed format> is a suggestion forthe correct format to use when parsing the data.

Some extensions in the ArcSight Extension Dictionary have names that start with the asterisk (*). SinceLogLogic Unity does not allow column names to start with asterisk (*), an asterisk (*) is omittedfrom the column name. For example, the *sourceProcessId extension is extracted into a columnnamed sourceProcessId.

When the event was written, the pipe (|), equal sign (=), and backslash (\) characters might havebeen escaped by inserting a backslash (\) in front of them. The CEF parser removes the backslash(\) character, returning the data to its original form. For example, if the value of the Name header inthe event is "detected a \| in message", the value of the cefName column will be "detected a | inmessage".

● Syslog Parser: Data conforming to the Syslog standard defined in RFC-5424 (https://tools.ietf.org/html/ rfc5424) can be parsed using the Syslog Parser.

The older, obsolete format described in RFC-3164 is not supported.

All the header fields defined in the format are extracted as is the Message component. If the log datacontains Structured Data elements, those are extracted as well with the names of the resultingcolumns being composed of <element-name>.<key name> as shown in the following example:<165>1 2003-10-11T22:14:15.003Z mymachine.example.com evntslog - ID47 [exampleSDID@32473iut="3" eventSource="Application" eventID="1011"] An application event log entry

108


https://protect724.hp.com/servlet/JiveServlet/previewBody/1072-102-6-4697/CommonEventFormat.pdf

https://tools.ietf.org/html/ rfc5424

https://tools.ietf.org/html/ rfc5424

The following columns are extracted:facility = local4; severity = notice; version = 1; timestamp = 2003-10-11 15:14:15 (if LogLogic Unity isrunning in the PDT time zone); hostname = mymachine.example.com; appname = evntslog; procid =<null>; msgid = ID47; [email protected] = 3; [email protected] = Application;[email protected] = 1011; msg = An application event log entry

Viewing Source ConfigurationsYou can view all defined source configurations, add new configurations, edit existing configurations,enable and disable configurations, and delete configurations.

On the toolbar, click the Administration icon, the Administration overview landing page displaysdifferent options. From the Sources panel, select the Configurations link. From the Configurationspage, you can perform the following tasks:

● Filter configurations

You can quickly find the desired configuration by typing the rule name in the Find field. As youstart typing a rule name in the Find field, the Configurations page is automatically refreshedshowing your selection.

● View configurations based on filters

You can use filters to easily find configurations. Click the View list to view different filters.

● Sort configurations

You can sort any column in ascending or descending order on the Configurations page. Click thecolumn name or click the arrow (that is displayed on the right side of the column name when youclick in that column) to sort the column.


You can show or hide columns, except the mandatory column, from the table . Click to view allavailable columns in the table. Select the check box to show the column. Clear the check box to hidethe column from the table. The Configurations page is updated immediately.

The Configurations page information is described below:

Column Description

Enable Indicates if the configuration is enabled or disabled

● ON indicates enabled.

● OFF indicates disabled.

All enabled configurations can be searched using the source filter on theSearch tab.

Name The name of the source configuration

Created by The name of the user who created the configuration

Date created The date when the configuration was first created

Last edited The date when the configuration was last updated

109


Adding a Source Configuration in Graphical ModeYou can add a source configuration that can be activated to analyze results in the normalized format. Ingraphical mode, a wizard helps you in adding a source configuration, source filter, parsing rule,previewing parser output, and modifying the rules.

Procedure


2. From the Administration overview landing page, click the Configurations link in the Sources panel.

3. From the Configurations page, click to add a configuration.By default, a graphical mode is opened. For instructions on how to add in raw mode, see Adding aSource Configuration in Raw Mode.

4. By default, the slider is set to ON to enable the configuration. Click the slider to OFF to disable theconfiguration.All enabled configurations can be searched using the source filter from the Search tab.

5. Enter the name of the source configuration in the Name field.The name must contain any alphanumeric character. It can also contain underscore (_) and hyphen(-).

6. Optional: Enter the description in the Description field.

7. Add a new source filter, for instructions see Defining a Source Filter.

8. Define a parsing rule, for instructions see Adding a Parsing Rule.

9. Manage your custom columns, for instructions see Managing Columns.

10. Click Save to add a new source configuration.The Configurations page is updated with the newly added configuration.

Defining a Source FilterYou can add a new source filter that is assigned to the source configuration. The source filters bindmultiple source configurations to a log source.

Procedure

1. In the Source filter field, enter the source filter statement that is assigned for this sourceconfiguration. Source filters can only be used on one or multiple system columns. All filterstatements as described in the Filter Statement section are supported, except that if a full text searchis desired, it must be specified explicitly, for example, sys_body CONTAINS '<searchstring>'.

For example, sys_sourceType=165 (device type ID that is retrieved from LMI) AND sys_bodyCONTAINS '<searchstring>'

If you specify multiple source configurations, the first configuration whose filter matcheswith the event is used to parse that event, extracting all columns specified by thatconfiguration.

2. Click Validate to validate the filter statement.

3. To add a new parsing rule, click 2. Add sample events and parsing rules or click located on theright side of the page.or,

To add only the source filter, click Save.

110


Adding a Parsing RuleYou can add one or multiple parsing rules that define how to parse log events.

Procedure

1. Paste the sample log data in the Sample events panel.This data can be helpful in defining the parsing rule based on the log source. Once you save thesource configuration, the sample data is always available when editing the same sourceconfiguration or associated parsing rules.

You can paste maximum of 100 KB sample data.

2. In the Parsing rules panel, click Add new rule to add a new parsing rule.You can add multiple rules for the same source configuration.

3. Enter the name of the rule in the Name field.The name must contain any alphanumeric character. It can also contain an underscore (_) andhyphen (-).

4. Make sure that the slider is set to ON to enable the parsing rule. Click the slider to OFF to disablethe parsing rule.

5. Enter the filter that is assigned to the parsing rule in the Filter field. All regular expression patternsare supported.

If you do not define the filter, all events are matched with this rule. If there are additionalrules after such parsing rule, then the additional rules are ignored.

6. From the Choose parser list, select the type of parser you want to use. The options are: Key-Value,Columnar, Regex, CEF and Syslog.

● For the Key-Value parser, define the following information:

— Values separator: Enter the delimiter that you want to use to separate key-value pairs. Youcan add only one separator at a time. The delimiters are case-sensitive. For example,user=bob,vm=windows where user=bob is one pair and vm=windows is another pairseparated with delimiter comma (,). The delimiter can be a single character, a string thathas to be matched exactly, or a Java regular expression.

RegEx: Select ON to use as a Java regular expression or OFF to use as a literal string.

111


— Key-value separator: Enter the delimiter that you want to use to separate keys from theirvalues. The delimiters are case-sensitive. For example, user=bob where user is a key andbob is a value separated with delimiter equal sign (=). The delimiter can be a singlecharacter, a string that has to be matched exactly, or a Java regular expression.

RegEx: Select ON to use as a Java regular expression or OFF to use as a literal string.

— Beginning (RegEx): If you want some initial characters in each line to be ignored, enter aregular expression for it. If a segment at the beginning of the line matches this regularexpression, it is ignored. For example, if a line starts with Login and then followed by key-value pairs, then if you enter Login in this field, the first word Login is ignored whenextracting columns. Named groups in the regular expression are extracted as columns.

— Ending (RegEx): If you want some characters at the end of each line to be ignored, enter aregular expression for it. If a segment at the end of the line matches this regular expression,it is ignored. Named groups in the regular expression are extracted as columns.

— Last key: Enter a key name. Whenever that key is found in a line, the parser stopssearching for more key-value pairs in that line and the value for that key is the remainingcontent of the line. For example, if the line endsSeverity="high",EventSubClass="1",ObjectID="389576426", then if you specifySeverity as the last key, then the value for severity is"",EventSubClass="1",ObjectID="389576426".

To specify a <space>, enter backslash s (\s) and for a <tab>, enter backslash t (\t).

● For the Columnar parser, define the following information:

— Separator: Enter the delimiter that you want to use as a column separator. The separatorcan be a string of one or more characters, or a Java regular expression. The delimiters arecase-sensitive. For example, bob,windows where comma (,) is a character used to separatetwo columns.

— RegEx: Use this option to define how the separator should be interpreted. Select ON to useas a Java regular expression or OFF to use as a literal string.

— Escape character: Define a character that is actually used to escape the character used as acolumn delimiter. The delimiters are case-sensitive. For example, if you use a comma as acolumn separator and your column value has a comma in it, then that value has to beescaped so that a parser does not think that the instance of the comma is the start of a newcolumn.

— Max columns: Enter the maximum number of columns to be extracted. If more columnsthan maxColumns are found, then the content of the additional columns is included in thelast column. For example, if the separator is <space> and the maxColumns value is 3 for amessage like “a b c d”, then there are 3 columns with values “a”, “b” and “c <space> d”.

— Trim values: If defined ON, then the extra (white) space that is generated at the beginningand end of the column is removed. If defined OFF, the extra space is not removed.

● For the Regex parser, define the following information:

— Regex pattern: Make sure to enter a valid PCRE regular expression that contains thegroups (named or unnamed) to extracted into column values from the log event. Also, it isgood practice to use one or more sample events to validate your regular expression andmake sure that the correct values are extracted from the event. For a list of supportedregular expression meta characters, based on Java regular expressions, see SupportedRegular Expression Characters.

For example,

112


(?<Sequence>\d+).*(?<ACL>\%\w+ \-\d\-\w+)\:\s(?<Name>\w+)\s(? <Version>\w+)\s(?<Status>\w+)\ s(?<Protocol>\w+)\s(?< SourceIP>\d{1,3}\.\d{1,3}\.\d{ 1,3}\.\d{1,3}).*(?< DestinationIP>\d{1,3}\.\d{1,3} \.\d{1,3}\.\d{1,3}).*

This extracts 8 fields: Sequence, ACL, Name, Version, Status, Protocol, SourceIP, andDestinationIP.

7. Click Auto generate columns to extract columns based on the parser type. All custom columns areextracted in the Manage columns for this rule panel. You can add, edit, and delete custom columns.Click to add a column. Click inside the Column and Expression fields to edit any values. Hover

over the row, and the Delete button is displayed on the right side of the row for you to deletethe column.

● Column: The name of the column that is displayed in the results. Click in the row to add orupdate any column name. The content assist shows contextual matches of the existing customcolumn names for you to select.

Two columns cannot have the same name. When defining column names, follow theguidelines described in the COLUMNS Statement section.

● Expression: Define how to map values extracted by parser into defined columns. You can usearithmetic operators and conversion functions when defining an expression. The conversionfunctions are typically used when you need to define new columns where the expressions fornew columns can use conversion functions to convert between data types and combine themusing various operators. For details about the arithmetic operators, see Filter Statement sectionand for conversion functions, see Predefined Functions. The type of expression depends on theparser type:

— For Key value parser, the expression uses a key name preceded with “$” to extract the valuefor the column. For example, $user is the value of the key "user" in the log line or null if thekey is not present.

— For Columnar parser, the expression uses the $<n> identifier where n is the column numberfor the value of column n. For example, $2 is the value of the column "2".

— For Regex parser, the columns are extracted using the capturing group pattern the namedcapturing group pattern or a combination of both.

If you select the parser and the column list is empty, the parser tries to guesscolumns from the sample data.

— For CEF parser, based on the ArcSight Extension Dictionary, the CEF header columns areextracted and the remaining data is formatted as key-value pairs. For example, Sep 1908:26:10 host CEF:0|Security|threatmanager|1.0|100|worm successfully stopped|10|src=10.0.0.1 dst=2.1.2.2 spt=1232

This extracts these columns and their values: $cefVersion=0, $cefDeviceVendor=Security,$cefDeviceProduct=threatmanager, $cefDeviceVersion=1.0, $cefSignatureID=100,$cefName=worm successfully stopped, $cefSeverity=10, $sourceAddress=10.0.0.1,$destinationAddress=2.1.2.2, $sourcePort=1232

8. Click to refresh the Parser preview panel to view all extracted columns and their data typesthat are matched by the corresponding parsing rule. Each event that matches with thecorresponding rule is identified in the same color for easy readability. For custom columns, click inthe Type field to change the supported data type. Select the data type from the list.

This option is available only when the data is pasted in the Sample events panel and atleast one parsing rule is enabled.

113


https://protect724.hp.com/servlet/JiveServlet/previewBody/1072-102-6-4697/CommonEventFormat.pdf

9. Click to add a new parsing rule.The Parsing rules panel displays the newly added rule.

10. Click 3. Review configuration or click located on the right side of the page to manage columns.For more information, see Managing Columns.

Editing Parsing RulesYou can update custom parsing rules at any time.

You cannot update parsing rules that are defined for the system source configuration and LogLogicUnity built-in source configurations.

Procedure

1. In the Parsing rules panel, click the rule name that you want to update.2. In the Edit parsing rule panel update the rule information.

For details about each field, see Adding a Parsing Rule.

3. Click to refresh the Parser preview panel to view all extracted columns and their data typesthat are matched by the corresponding parsing rule. Each event that matches with thecorresponding rule is identified in the same color for easy readability.

114


4. Click to save the updated information.

The Parsing rules panel is updated immediately.

Defining Parsing Rules OrderWhen there are multiple parsing rules defined for a single source configuration, you can set the ruleorder. All columns are extracted as per the first rule definition that matches the event. For example, ifRule1 matches some of your data then it will be used to extract column values. If Rule1 fails to matchwith your data, then only Rule2 is applied, and so on.

Procedure

1. In the Parsing rules panel, hover over the rule row near the drag icon and the cursor turns intoa hand , which you can use to drag the row up or down to change the order.The Parsing Rules panel is updated immediately.

2. Click to refresh the Parser preview panel to view all extracted columns and their data typesthat are matched by the corresponding parsing rule. Each event that matches with thecorresponding rule is identified in the same color for easy readability.

3. Click Save to save the configuration.

Copying Parsing RulesYou can copy the same parsing rule as a new rule.

Procedure

1. In the Parsing rules panel, hover over the rule row and the Copy button is displayed on the

right side of the row. Click to save the same rule.The Parsing rules panel is updated immediately showing a newly added rule, for example,copy_rulename.

2. Click Save.

Deleting Parsing RulesYou can delete parsing rules from the system.

You cannot delete parsing rules that are defined for the system source configuration and LogLogic Unitybuilt-in source configurations.

Procedure

1. In the Parsing rules panel, hover over the rule row and the Delete button is displayed on the

right side of the row. Click to delete the parsing rule.The Parsing rules panel is updated immediately.

2. Click Save to save the configuration.

115


Managing ColumnsFrom the Review configuration page, you can update columns and data types for the associated sourceconfiguration. You can also review column statistics for each defined parsing rule.

Procedure

1. In the Columns panel all system and custom columns are displayed . You can add or removecolumns by selecting the check box that is next to the column name. You can update any columnname and type.

● Name: The name of the column that is displayed in the results. Click in the row to add or updateany column name. There are no restrictions on the characters used in column names, but if thecolumn name contains non-alphabetic or non-digit characters it need to be enclosed in brackets([ ]) when used in a search query or an expression.

Two columns cannot have the same name.

● Type: The data type of the column. Click in the column to add or update the supported datatypes. Select the data type from the list.

● Parser rules: The rule name that includes the defined column.

2. Select the Show system columns check box to show all system columns. By default, some systemcolumns are selected. If the check box is not selected, only the user defined columns and somedefault system columns are displayed. For a list of system columns, see About Columns.

3. After modifying column list, click to refresh the Parser preview panel to view all extractedcolumns and their data types for the defined parsing rule. For custom columns, click in the Typefield to change the supported data type. Select the data type from the list.

4. Click Save to save the column updates.

The Match statistics panel helps you view an overall information about events that are matched byspecified parsing rule. It displays how many rules are enabled, how many columns are extracted bythe rule, and how many events are matched with each rule.

116


Adding a Source Configuration in Raw ModeYou can add a new source configuration that can be activated to analyze results in the normalizedformat. All enabled configurations can be searched using the source filter from the Search tab.

Prerequisites

This option is for advanced users who understand JSON syntax to create a new parsing rule. If not, usethe graphical mode to create new source configuration. For details, see Adding a Source Configurationin Graphical Mode.

Procedure



3. From the Configurations page, click to add a new configuration.4. Click Switch to raw mode to add a new configuration in raw mode.5. In the Sample events panel, paste the sample events to analyze data in normalized format.

This data can be helpful in defining the parsing rule based on the log source. Once you add thesource configuration, the sample data is always available when editing the same sourceconfiguration or associated parsing rules.

You can paste maximum of 100 KB sample data.

6. In the Raw configuration mode panel, enter the parsing rule.Make sure to define source filter, parsing rule, and parser properties in a valid JSON syntax, asshown below:{ "sourceConfig": { "name": "SourceConfiguration_1", "active": "true", "sourceFilter": "", "parsingRules": [], "columns": [ { "name": "sys_domain", "type": "STRING" }, { "name": "sys_eventTime", "type": "TIMESTAMP" }, { "name": "sys_body", "type": "STRING" }, { "name": "sys_bodySize", "type": "INT" }, { "name": "sys_collectTime", "type": "TIMESTAMP" }, { "name": "sys_sourceType", "type": "INT" }, { "name": "sys_collectIP",

117


"type": "INET_ADDR" }, { "name": "sys_sourceDnsName", "type": "STRING" }, { "name": "sys_filename", "type": "STRING" }, { "name": "sys_collectIPZone", "type": "STRING" } ] }}

For sample parsing rules in JSON format for each parser type, see Parsing Rule JSON syntax.

7. Click Validate to ensure that the rule syntax is valid. Click Format to format the JSON.

8. Click to refresh the Parser preview panel to view all extracted columns and their data typesthat are matched by the defined parsing rule. Click in the Type field to change the supported datatypes and select the data type from the list.

This option is available only when the data is pasted in the Sample events panel and atleast one parsing rule is enabled.

9. Click Save to add a new source configuration.The Configurations page displays the newly added configuration.

Enabling or Disabling Source ConfigurationsSource configurations can be enabled or disabled at any time. All enabled configurations can besearched using the source filter on the Search tab.

By default, the system source configuration is enabled. You cannot disable the system sourceconfiguration.

Procedure



3. From the Configurations page, click the slider in the Enable column to ON to enable the sourceconfiguration.

4. From the Configurations page, click the slider in the Enable column to OFF to disable the sourceconfiguration.

Editing Source ConfigurationsYou can update existing configurations at any time. You can save the same configuration as a newconfiguration.

You cannot update the system source configuration and LogLogic Unity built-in source configurationsfrom the system. See Supported Log Sources list for details.

118


Procedure


2. From the Administration overview landing page, click the Configurations link in the Sources panel.3. From the Configurations page, click the configuration name that you want to update.

The Details panel opens on the right side of the page.4. Click the Edit link to update the configuration.

For detailed information, see Adding a Source Configuration in Graphical Mode.

5. Click to refresh the Parser preview panel to view all extracted columns and their data typesthat are matched by the corresponding parsing rule. Each event that matches with thecorresponding rule are identified in the same color for easy readability.

6. Click Save to save the updated information.

The Configurations page is updated immediately.7. Click Save As to save the same configuration as a new source configuration. Enter the new source

configuration name in the Name field and click Ok.

The Configurations page is updated immediately showing the newly added source configuration.

Duplicating Source ConfigurationsYou can copy the imported and system generated source configurations, except the LogLogic Unitybuilt-in source configurations, as a new source configuration that you allows you to modify as per yourneed.

Procedure


2. From the Administration overview landing page, click the Configurations link in the Sources panel.3. From the Configurations page, select the source configuration that you want to copy by selecting the

check box located next to the Enable column and click to copy the source configuration.

The Duplicate button is enabled after you select a source configuration from the list.4. Enter the new name in the Name field and click OK.

The newly added source configuration is displayed on the Configurations page immediately.

Deleting Source ConfigurationsYou can delete one or multiple custom configurations from the system. Once you delete anyconfiguration, it cannot be recovered.

You cannot delete the system source configuration and LogLogic Unity built-in source configurationsfrom the system. See Supported Log Sources list for details.

Procedure


119



3. From the Configurations page, select the check box located next to the configuration name that you

want to delete and click .

4. In the confirmation window, click Ok to delete the configuration from the system.The Configurations page is updated immediately.

120


Manage Sources

You can forward all events from LogLogic LMI appliance to LogLogic Unity. From the Sources tab, youcan view all sources that are exported into LogLogic Unity. Once all sources are exported, you cansearch for events from a specific source on the Search tab.

By default, sources are exported to the LogLogic Unity system every 15 minutes. You can change thedefault export interval using the ./llconf file. For instructions, see Setting up the Export Interval.

Prerequisites

● Access the LogLogic LMI v5.6.1 instance.

● Create a new Outbound Data rule or use All Sources rule to specify log sources that should beexported to LogLogic Unity and define LogLogic Unity as the Destination IP. For details about howto define the message routing rule on the LogLogic LMI appliance, refer to the TIBCO LogLogic® LogManagement Intelligence Administration Guide.

For example, if you export Blue_Coat_Syslog source, you can run the following search queries to seeevents retrieved from the specific source for the defined time period:

use Blue_Coat_Syslog | sys_source = "::ffff:1.0.0.1_ bluecoatsyslog" | sys_eventTime

in -10y

sys_source = "::ffff:1.0.0.1_ bluecoatsyslog" | sys_eventTime in -10y

sys_source like '%SQL%' | sys_eventTime in -10y

sys_source_description in ('Apache Web Server Access','vShield Edge') |

sys_eventTime in -10y

Similarly, you can run the following correlation queries to see events retrieved from the specific sourcefor the defined time period:

use Blue_Coat_Syslog Within 1h Event Group SG1 where sys_source = "::ffff:1.0.0.1_

bluecoatsyslog"

use Blue_Coat_Syslog Within 1d Event Group SG2 where sys_source in ( "::ffff:

5.0.100.100_ bluecoatsyslog", "::ffff:5.0.100.101_ bluecoatsyslog", "::ffff:

5.0.100.102_ bluecoatsyslog")

Viewing SourcesYou can view all defined log sources, add new sources, edit sources, enable and disable sources, anddelete sources.

On the toolbar, click the Administration icon, the Administration overview landing page displaysdifferent options. From the Sources panel, select the Sources link. From the Sources page, you canperform the following tasks:

● Filter sources

You can quickly find the desired source by typing the source name in the Find field. As you starttyping a source name in the Find field, the Sources page is automatically refreshed showing yourselection.

● View source details

You can view the source details by selecting the source row. The Details panel opens on the right

side of the page showing the detailed information of the source. Click to close the Details panel.

● View sources based on filters

121


You can use filters to easily find sources that are exported in the system. Click the View list to viewdifferent filters.

● Sort sources

You can sort any column in ascending or descending order on the Sources page. Click the columnname or click the arrow (that is displayed on the right side of the column name when you click inthat column) to sort the column.


You can show or hide columns from the table. Click to view all available columns in the table.Select the check box to show the column. Clear the check box to hide the column from the table. TheSources page is updated immediately.

The Sources page information is described below:

Column Description

Name The name of the source

Source type The type of the source configuration

IP address The IP address of the source

Collect IP Zone The identification name that is used to identify each message sent from aparticular source

Description The description of the source

Concentrator ID The IP address of the LogLogic LMI appliance from where the source issynchronized

Last updated The last updated date when the LMI appliance was synchronized

Setting up the Export IntervalBy default, sources are exported to LogLogic Unity every 15 minutes. You can change the default exportinterval on each LogLogic Unity system.

Procedure

1. Create a configuration template as shown below.{ "configurations": [ { "lmiExport": { "exportInterval": 5 } }]}

2. Specify a new exportInterval value. For example, 5 minutes, as shown above.


4. Run the following command to upload the configuration file:./llconf [-c <config server host:port>] -f <config file path>

122


Manage Domains

You can manage your domains or add a separate domain for analyzing events without disturbing arunning instance.

The Domains menu is only available to administrators (users with administrator privileges).

A domain is an area of storage for events and their associated data. You can split data of differentnature or intended usage into multiple domains. You can search for events from a specific domain ormultiple domains from the Search tab. Similarly, you can specify which data should be used foralerting. You can manually delete a domain and reclaim the disk space used by that domain.

The following three pre-defined domains are created:

● shared: a default single general purpose domain used in searching and alerting.

● internal: an internal domain to store LogLogic Unity events. For example, you can run the followingsearch query to see all events from all nodes (Data node, Query node, Event Distributor, Correlationnode, Web node, and Configuration node) within last one day:

use unity | sys_domain = 'internal'| sys_eventTime IN -1d

● samples: a domain to store sample events that are provided with the product.

Viewing DomainsYou can view all domains, add a new domain, edit, or delete domains.

On the toolbar, click the Administration icon, the Administration overview landing page displaysdifferent options. Select the Domains link. From the Domains page, you can perform the followingtasks:

● Filter domains

You can quickly find the desired domain by typing the domain name in the Find field. As you starttyping a domain name in the Find field, the Domains page is automatically refreshed showing yourselection.

● View domains based on filters

You can use filters to easily find domains in the system. Click the View list to view different filters.

● Sort domains

You can sort any column in ascending or descending order on the Domains page. Click the columnname or click the arrow (that is displayed on the right side of the column name when you click inthat column) to sort the column.


You can show or hide columns, except the mandatory column, from the table . Click to view allavailable columns in the table. Select the check box to show the column. Clear the check box to hidethe column from the table. The Domains page is updated immediately.

The Domains page information is described below:

Column Description

Name The domain name.

123


Column Description

Description The description of the domain.

Created by The name of the user who created the domain.

Date created The date when the domain was first created.

Last modified The date when the domain was last updated.

Adding a DomainYou can add a new domain that can be used for importing or forwarding events.

Procedure


2. From the Administration overview landing page, click the Domains link.

3. From the Domains page, click to add a new domain.

4. Enter the domain name in the Name field.The name can have maximum 20 characters but cannot contain capital letters and special characters,for example, *?:[]\"<>|(){}&'!;/


6. Enter the number of copies of an event to be stored in the LogLogic Unity system in the Replicationfactor field.Replication factor controls how many copies of an event ingested by LogLogic Unity will be storedon Data nodes. The default value of 1 means there is just one single copy and losing a Data nodewill result in losing all the data stored on it. With a replication factor >= 2, additional copies aremade that allows for the loss of a Data node without losing the data due to the redundancyprovided with the additional copies. Setting the value to <n> implies a resilience to the loss of atmost <n-1> Data node. If the Replication factor is larger than the number of available data nodes, awarning is displayed.

7. Select the Storage zone from the list that can be associated with the domain.The StorageZone configuration helps you define how data is distributed among Event Distributor(ED) nodes and data nodes. Each domain is assigned to a StorageZone, each ED node and data nodeare also assigned to a StorageZone.

8. Select the Retention policy from the list that can be associated with the domain.The Default retention policy is set for 90 days. An admin can add a new retention policy. For details,see Adding a Retention Policy.

9. By default, the Allow this domain to be searchable by default check box is selected. This optionallows to search all events that are stored in this domain.

If this option is not selected, you must specify a domain name in your search query forevents to be retrieved from the specific domain as shown in the following example:

use system | sys_domain = 'mylogspace' | sys_eventTime IN -1w

10. By default, the Allow this domain to be used for generating alerts check box is selected. Thisoption allows this domain to generate alerts.If this option is not selected, alerts will not be generated from this domain.

124


11. Click Save to add a new domain.The newly added domain is displayed on the Domains page.

Editing a DomainYou can only update domains that you have created. An admin user can edit any domain. By default,the shared and internal domains cannot be edited.

Procedure



3. From the Domains page, select the domain name that you want to update.The Details panel opens on the right side of the page. For information about replication factor, referto the Adding a Domain section. It also displays the used disk space for that domain and thepartition information. Click on the Partitions arrow (>) to view the details. Click the More link toview the partition details. In the Datanodes column if the data node is a leader, it is displayed as (L)and if the data node is in error state, it is displayed as (E).

4. Click the Edit link to update the domain information. Make the necessary updates.

5. Click Save to save the updated information.The updated domain is displayed on the Domains page.

Deleting a DomainYou can only delete domains that you have created. An admin user can delete any domain. By default,the shared and internal domains cannot be deleted.

Procedure



3. From the Domains page, select the domain that you want to delete by selecting the check box

located next to the Name column and click . To select all domains in the system, except thedefault domains, select the check box located next to the Name column header.

The Delete button is enabled after you select one or more domains. For the defaultdomains shared and internal, check boxes are disabled.

4. In the confirmation window, click Ok to delete the domain and all of its content from the system.The Domains page is updated immediately.

125


Manage Data Nodes

You can view which data nodes are hosting partitions for a given domain.

The Datanodes menu is only available to administrators (users with administrator privileges).

Viewing Data NodesYou can easily monitor all data nodes in the system.

On the toolbar, click the Administration icon, the Administration overview landing page displaysdifferent options. Select the Datanodes link. From the Datanodes page, you can perform the followingtasks:

● Filter data nodes

You can quickly find the desired datanode by typing the datanode identification number in the Findfield. As you start typing a datanode ID in the Find field, the Datanodes page is automaticallyrefreshed showing your selection.

● View data nodes based on filters

You can use filters to easily find data nodes in the system. Click the View list to view differentfilters.

● Sort data nodes

You can sort any column in ascending or descending order on the Datanodes page. Click the columnname or click the arrow (that is displayed on the right side of the column name when you click inthat column) to sort the column.


You can show or hide columns, except the mandatory column, from the table . Click to view allavailable columns in the table. Select the check box to show the column. Clear the check box to hidethe column from the table. The Datanodes page is updated immediately.

● View details

Click on the data node row to view the details on the right side. The data node information isdisplayed in the Details pane. Click on the Replicas arrow (>) to view the details. Click the Morelink to view the replicas details.

The Datanodes page information is described below:

Column Description

Datanode id The data node identification number.

Domains The number of domains on this data node.

Storage zone The name of the storage zone.

Status The status of the data node.

126


Manage Users

Using the User management page, an admin can manage users, groups, roles, and permissions.

Users are entities that need access to the system. User roles determine the permissions the user has forperforming system operation tasks. By default, LogLogic Unity comes with the following rolespredefined:

● admin: this role has the most capabilities assigned to it and is intended for administrators who willmanage all or most of the users and groups. This role will also be able to run searches, managetriggers, Bloks, or sources.

Your system must have at least one enabled user with the admin role.

● user: this role can run searches, manage triggers, Bloks, or sources.

When a user is selected the users information is displayed in a User Card on the right.

LogLogic Unity can be configured to accept user accounts that can log into a defined LogLogic LMIinstance, see Enabling LMI Users to access LogLogic Unity.

Viewing UsersAn admin can manage the users authorized to use LogLogic Unity. An admin can add new users,disable user accounts, delete users from the system, or edit user information.

On the toolbar, click the Administration icon, the Administration overview landing page displaysdifferent options. From the Users panel, select the Users link. From the Users page, you can do thefollowing:

● Filter users

You can quickly find the desired user by typing the user name in the Find field. As you start typinga user name in the Find field, the Users page is automatically refreshed showing your selection.

● View users based on filters

You can use filters to easily find users in the system. Click the View list to view different filters.

● Sort users

You can sort any column in the ascending or descending order on the Users page. Click the columnname or click the arrow (that is displayed on the right side of the column name when you click inthat column) to sort the column.


You can show or hide columns, except the mandatory column, from the table . Click to view allavailable columns in the table. Select the check box to show the column. Clear the check box to hidethe column from the table. The Users page is updated immediately.

The Users page information is described below:

Field Description

Enable Indicates if the user is enabled or disabled

User name Mandatory field — A unique identifier for each user

127


Field Description

First name The user's first name

Last name The user's last name

Status Indicates if the user is online or offline

Role The assigned role of the user. It can be either User or Admin. This field willbe empty if the user is not assigned to a role. A user who is not assigned arole will not be able to login to LogLogic Unity.

Last sign-in The time the user last signed in

Adding a UserOnly an admin can add new users.

Procedure


2. From the Administration overview landing page, click the Users link in the Users panel.

3. From the Users page, click to add a new user.

4. In the Add User window, enter the following information:

● User name: Enter the name of a user.

● Email: Optional, enter user's email address.

● First name: Optional, enter user's first name.

● Last name: Optional, enter user's last name.

● Phone: Optional, enter user's phone number.

● Password: Enter the user password. It is recommended that users change their passwords thefirst time they login.

● Verify password: Re-enter the user password.

● Group: Optional, from the drop-down menu select a group. Users can be assigned to multiplegroups.

● Role: From the drop-down menu select a role.

● Enable: Slide to ON to enable the user. If this is not selected the user will not be able to login.

Disabled users or users who have not been assigned a role will not be able to login.

5. Click Save to add the new user in the system.The newly added user is listed in the Users page.

128


Editing User InformationAn admin can edit all user information, but users can only edit their own information via Edit Profile.

Procedure



3. From the Users page, click on a user name.The users details are displayed on the right.

4. Select Edit from the User card located on the right.

5. In the Edit User Info window, edit the fields you want to update.


Disabling a UserOnly an admin can disable a user.

Procedure



3. From the Users page, click on a user name.The users details are displayed on the right.

4. Select Edit from the user card located on the right.

5. In the Edit User Info page slide the Enable slider to OFF.


Disabled users are not able to login to the system.

A user can also be disabled via the User management page by deselecting the Enabledslider next to the user's name.

Deleting UserOnly an admin can delete a user.

In order for LogLogic Unity to work, your system must contain at least one admin user.

Procedure



3. From the Users page, select the checkbox next to the user that you want to delete and click .

129


4. In the confirmation window, click OK to delete the user from the system.The Users page is updated.

Once users are deleted their information cannot be recovered.

Enabling LogLogic LMI Users to Access LogLogic UnityAn admin can configure LogLogic Unity system to accept user accounts that can log into a definedLogLogic LMI instance using Active Directory (AD) or Terminal Access Controller Access-ControlSystem (TACACS) authentication protocol. When these remote users run a search query in LogLogicUnity, they can only view results from the sources they have access in the LogLogic LMI instance.

Prerequisites

● Access the LogLogic LMI v5.6.1 instance.● Create a new Outbound Data rule or use All Sources rule to specify log sources that should be

exported to LogLogic Unity and define LogLogic Unity as the Destination IP. For details about howto define the message routing rule on the LogLogic LMI appliance, refer to TIBCO LogLogic® LogManagement Intelligence Administration Guide.

● Set up a remote authentication access to the LogLogic LMI host machine. For instructions, see Setting up LogLogic LMI Authentication.

Users that are defined in AD or TACACS are not visible in the User management > Users page. Youmust configure the user group membership and password on the AD or TACACS host machines. Also,set up all configuration related to the definition of which devices a given group may see in a result seton the LogLogic LMI instance (from the Management > Users > Directory roles menu). For instructionson how to set up user roles and privileges, refer to TIBCO LogLogic® Log Management IntelligenceAdministration Guide.

A special case of dynamic group, in case of multiple LogLogic LMI instances sending their events to thesame LogLogic Unity system, must be considered. The definition of dynamic groups will be applied tothe events coming from ALL the LogLogic LMI instances, and not just the one that holds the definitionof the dynamic group. Because of that if dynamic groups are used on LogLogic LMI, the resulting set ofevents that can be accessed in LogLogic Unity and LogLogic LMI may be different.

For example, if on LMI_1 a dynamic group DG_1 is defined that matches Windows events and onLMI_2 has another dynamic group DG_2 matching Cisco IOS events. If a remote user is part of adirectory role on LMI_1 that contains DG_1 and part of a directory role on LMI_2 that contains DG_2,then that remote user can view all Windows AND Cisco IOS events from BOTH LMI_1 and LMI2instances. However, he can still only view Windows events on LMI_1 and Cisco IOS events on LMI_2instance respectively.

If this behavior is not intended, it is a good practice to use global groups for a precise definition ofremote user roles in LogLogic Unity, across multiple LMI instances.

If multiple LogLogic LMI instances are forwarding rules to the same LogLogic Unity system, it meansthey are configured with the same AD or TACAS authentication settings so the authentication takesplace only against one single LogLogic LMI instance. LogLogic Unity supports global groups definedby the management station when used in directory roles configuration.

All remote users do not have the admin role on the LogLogic Unity system. Only LogLogic Unityadmin users can access the User management page. When a given user account is defined both locallyon LogLogic Unity and through LogLogic LMI authentication, the local user account is used. LocalLogLogic Unity accounts have no restriction in terms of the devices that are visible in a search query.

When LogLogic Unity admin users access the User management page, a banner is displayed on theupper-right corner showing that the system is configured to accept user credentials valid forauthentication from the specified LogLogic LMI instance as shown below.

130


The Admin > Sources > Source management > Sources page lists all exported LMI log sources, eventhose for which the current user does not have access to the log data. For details, see Manage Sources.

Setting up Remote AuthenticationYou must setup LogLogic LMI remote authentication for LogLogic Unity by uploading theconfiguration file.

Procedure

1. Create a configuration file as shown below.{ "configurations": [ { "remoteAuth": { "auth": { "host": "<LogLogic LMI IP address>" } } } ]}

2. Specify a "host": "<LogLogic LMI IP address>" value. For example, 192.168.1.xxx.


4. Run the following command to upload the configuration file:llconf -c <config server host:port> -f <config file path>

131


Manage Groups

Using the User management page, an admin can create groups to manage a collection of users.

A user group is a collection of user accounts and can have multiple roles assigned to it. When a user isassigned to a group, the group's roles are automatically assigned to the user. Users can be associatedwith multiple groups.

Viewing GroupsAn admin can create groups to manage a collection of users. Add a new group, disable a group, deletea group from the system, or edit group information.

On the toolbar, click the Administration icon, the Administration overview landing page displaysdifferent options. From the Users panel, select the Groups link. From the Groups page, you can do thefollowing:

● Filter groups

You can quickly find the desired group by typing the group name in the Find field. As you starttyping a group name in the Find field, the Groups page is automatically refreshed showing yourselection.

● View groups based on filters

You can use filters to easily find user groups in the system. Click the View list to view differentfilters.

● Sort groups

You can sort any column in the ascending or descending order on the Groups page. Click thecolumn name or click the arrow (that is displayed on the right side of the column name when youclick in that column) to sort the column.


You can show or hide columns, except the mandatory column, from the table . Click to view allavailable columns in the table. Select the check box to show the column. Clear the check box to hidethe column from the table. The Groups page is updated immediately.

The Groups page information is described below:

Field Description

Active Indicates if the group is active or inactive

Users can be assigned to groups that are not active

Group The unique identifier for each group

Description Provides a description of the group

Role The role assigned to all users within the group

132


Adding a GroupOnly an admin can add new groups.

Procedure


2. From the Administration overview landing page, click the Groups link in the Users panel.

3. From Groups page, click to add a new group.

4. In the Add Group window, enter the following information:

● Group name: Enter a name for the group.● Description: Optional, enter a description of the group.● Role: Optional, enter the role that will be assigned to all users in this group.● Add users: Optional, enter the names of all the users that will be connected to this group. Names

should be separated by a comma.● Enable: Slide the Enable slide to ON to enable the group.

5. Click Save to add the new group in the system.The newly added group is listed in the Groups page.

Editing a GroupAn admin can edit all group information.

Procedure



3. From Groups page, click on a group.The group details are displayed on the right.

4. Select Edit from the Group card located on the right.

5. In the Edit Group Info window, edit the fields you want to update.


Disabling a GroupOnly an admin can disable a group.

Procedure



3. From Groups page, click on a group.The group details are displayed on the right.

133


4. Select Edit from the Group card located on the right.

5. In the Edit Group Info window, slide the Enable slider to OFF.


Users can be assigned to inactive groups.

A group can also be disabled via the Groups page by deselecting the Enabled slider nextto the group's name.

Deleting a GroupOnly an admin can delete a group.

Procedure



3. From Groups page, select the group that you want to delete and click .

4. In the confirmation window, click Ok to delete the group from the system.The Groups account page will be updated.

Once a group is deleted its information cannot be recovered.

134


Manage System

Using the System menu, an admin can maintain control and visibility into the LogLogic Unitydeployment and easily make changes or adjustments to the configurations.

The System menu is only available to administrators (users with administrator privileges).

Manage Retention PoliciesAn admin can manage retention policy that controls both data retention location and time so that theadmin can control how much data is being consumed by the application. The Default retention policy isset for 90 days and cannot be deleted.

Viewing Retention PoliciesAn admin can view all policies, add a new policy, edit, or delete retention policies in the system.

On the toolbar, click the Administration icon, the Administration overview landing page displaysdifferent options. From the System panel, select the Retention link. From the Retention page, you canperform the following tasks:

● Filter policies

You can quickly find the desired policy by typing the policy name in the Find field. As you starttyping a policy name in the Find field, the Retention page is automatically refreshed showing yourselection.

● View policies based on filters

You can use filters to easily find policies in the system. Click the View list to view different filters.

● Sort policies

You can sort any column in ascending or descending order on the Retention page. Click the columnname or click the arrow (that is displayed on the right side of the column name when you click inthat column) to sort the column.


You can show or hide columns, except the mandatory column, from the table . Click to view allavailable columns in the table. Select the check box to show the column. Clear the check box to hidethe column from the table. The Retention page is updated immediately.

The Retention page information is described below:

Column Description

Name The retention policy name.

Data retention time The duration until the data is retained.

Raw data size The size of raw data.

Index data size The size of indexed data.

135


Adding a Retention PolicyAn admin can define a new retention policy that controls the data location and the retention time.

Procedure


2. From the Administration overview landing page, click the Retention link in the System panel.

3. From the Retention page, click to add a new policy.

4. Enter the policy name in the Name field.


6. Enter the retention time in the Data retention time field.You can add time in days, months, and years. The time must start with the minus sign followed by anumber, and then one character from the set d,D,m,M,y,Y. For example, -2d means 2 days, -3mmeans 3 months, -4Y means 4 years.

7. In the Domains panel, located on the right side, click the check box to select the domain that shouldbe associated with the retention policy. You can also select and drag a specific domain into theDomains field.The selected domains are displayed in the Domains field.

8. Click Save to add a new policy.The newly added retention policy is displayed on the Retention page.

Editing a Retention PolicyAn admin can only update retention policies. The Default retention policy cannot be edited.

Procedure



3. From the Retention page, select the policy name that you want to update.The Details panel opens on the right side of the page.

4. Click the Edit link to update the policy information. Make the necessary updates.For information about each field, see Adding a Retention Policy.

5. Click Save to save the updated information.The updated policy is displayed on the Retention page.

Deleting a Retention PolicyAn admin can delete policies except the Default policy. Similarly, a retention policy that is associatedwith the domain cannot be deleted.

Procedure



136


3. From the Retention page, select the policy name that you want to delete by selecting the check box

located next to the Retention policy name column and click . To select all policies in the system,select the check box located next to the Retention policy name column header.

The Delete button is enabled after you select one or more policies.

4. In the confirmation window, click Ok to delete the retention policy from the system.The Retention page is updated immediately.

137


Manage Collectors

Using the Collectors menu, an admin can configure various data sources from which machine data canbe collected and fed into LogLogic Unity for further analysis. You can configure and manage day-to-day operation of the collector.

The Collectors menu is only available to administrators (users with administrator privileges).

By default, a disabled DefaultHawkCollector is visible in the system. Make sure to enable the collector forexisting Hawk deployments that have correct message transport settings configured. If you do not haveany existing Hawk deployments, you must manually configure the Hawk connector node. Forinstructions on how to configure the Hawk connector node, see TIBCO LogLogic® Unity Installation andConfiguration guide.

You can collect data using the following types of collectors:

● TIBCO Hawk®: is an event-based monitoring system built for managing distributed applicationsand operating systems. TIBCO Hawk uses TIBCO Messaging software for communication andinherits many of its benefits. You can choose one of the following as the primary message transportmechanisms to communicate between TIBCO Hawk deployment and LogLogic Unity:

— TIBCO Rendezvous®

— TIBCO Enterprise Messaging Service™

The Hawk collector only subscribes to those methods that have no parameters.

● Syslog: is a way for network devices to send events to a Syslog server. Syslog events can be collectedusing the TCP or UDP protocols. You must specify a binding host address and the port number ofthe syslog server that connects with the LogLogic Unity system.

● TIBCO ActiveMatrix BusinessWorks™: provides a flexible framework that allows you to scale yourruntime environment as needed. Using the TIBCO LogLogic® Unity Plug-in for ActiveMatrixBusinessWorks™, configure ActiveMatrix BusinessWorks (BW) to connect to the LogLogic Unitysystem. Once the connection is established, the BW can publish events generated by the activitiesexecuted in BW 6.3.1 to the LogLogic Unity system.

Viewing CollectorsAn admin can view all collectors, add a new, edit, or delete data collectors in the system.

On the toolbar, click the Administration icon, the Administration overview landing page displaysdifferent options. Select the Collectors link. From the Collector management page, you can perform thefollowing tasks:

● Filter collectors

You can quickly find the desired collector by typing the collector name in the Find field. As you starttyping a collector name in the Find field, the Collector management page is automatically refreshedshowing your selection.

● View collectors based on filters

You can use filters to easily find collectors in the system. Click the View list to view different filters.

● Sort collectors

You can sort any column in ascending or descending order on the Collector management page.Click the column name or click the arrow (that is displayed on the right side of the column namewhen you click in that column) to sort the column.

138



You can show or hide columns, except the mandatory column, from the table . Click to view allavailable columns in the table. Select the check box to show the column. Clear the check box to hidethe column from the table. The Collector management page is updated immediately.

The Collector management page information is described below:

Column Description

Enable The status of the collector. Click the slider to change the collector status.

● ON: Enabled

● OFF: Disabled

Name The name of the collector.

Type The type of the collector.

Description The description of the collector.

Total messages The total number of messages collected.

Last message received The time and date when the last event was received in the collector.

Adding a Hawk CollectorLogLogic Unity must connect to any existing TIBCO Hawk deployment via local or remote connectionso that a new Hawk collector can be used for storing data.

Prerequisites

● TIBCO Hawk 4.9.1, 5.1, 5.1.1 or 5.2 must be installed and configured to connect via local or remoteconnection to LogLogic Unity.

For more details about TIBCO Hawk, refer to the TIBCO Hawk documentation.

Procedure


2. From the Administration overview landing page, click the Collectors link.

3. From the Collector management page, click to add a new collector.

4. Enter the collector name in the Name field.

5. From the Type of collector list, select Hawk.


7. Enter the Hawk domain name in the Hawk Domain field.

8. In the Message Transport list, select the transport option by which monitoring data is collected intothe LogLogic Unity system.For detailed information about each option, refer to the Using TIBCO Rendezvous and UsingTIBCO EMS sections.

139


9. Enter the time in seconds in the Subscription interval field.

10. Select the Enable Hawk Alert/Subscription Data Collection check box to enable the TIBCO Hawkdata (Alerts, Subscription, and Events) collection into the LogLogic Unity system.

11. Select from the Unity Domain list of pre-configured domains where events should be stored ordefine a new domain by clicking the Create new domain link. For information on how to create anew domain, see Adding a Domain.

12. Click Test to test the connection.After the successful connection, the new collector can be added.

13. Click Save to add a new collector.The newly added collector is displayed on the Collector management page.

Using TIBCO Rendezvous (RV) Message TransportTIBCO Rendezvous (RV) allows distributed applications to exchange data across a network. TIBCO RVprovides software applications robust support for network data transport and network datarepresentation.

Procedure

1. From the Message Transport list, select TIBCO Rendezvous (RV).

2. Define the following fields:

Field Description

Service This instructs the Rendezvous daemon to use this service whenever itconveys messages on this transport. You can specify the port number as theservice to be used, for example, "7474".

Network This instructs the Rendezvous daemon to use a particular network for allcommunications involving this transport. The network parameter consistsof up to three parts, separated by semicolons: network, multicast groups,and send address.

Daemon This instructs the transport creation function about how and where to findthe Rendezvous daemon and establish communication. For remotedaemons, specify two parts (introducing the remote host name as the firstpart), for example, tcp:7474: Remote host name; Port number.





Using TIBCO Enterprise Message Service (EMS) Message TransportTIBCO Enterprise Messaging Service (EMS) is based on Java Message Service (JMS), the messagingspecification of the J2EE (Java Platform Enterprise Edition) architecture. It provides a standardized

140


interface for enabling communications between J2EE-compliant applications, Enterprise Java Beans,and various application servers.

Procedure

1. From the Message Transport list, select TIBCO Enterprise Message Service (EMS).

2. Define the following fields:

Field Description

EMS Server The URL of the EMS server.

User Name The valid username used to connect to the EMS server.

Password The password used to connect to the EMS server.

SSL Configuration Click the check box to specify the SSL parameters. After you select thecheck box, the following parameters are displayed.

Vendor Select the name of the SSL vendor from the list.

Trace Select the option from the list to enable the trace: True or False.

Trusted The option specifies the file name of the server certificates. This option canbe repeated if more than one certificate file is used.

Private Key The option indicates the private key used by the TIBCO Hawk component.

Private Keyencoding

Encoding is applied on the private key of the SSL certificate.

Password The password to decrypt the identify file of the TIBCO Hawk component.

Expectedhostname

The name the TIBCO Hawk expects in the CN field of the server’scertificate. If this parameter is not set, the expected name is the hostname ofthe server.

Identity The option specifies the digital certificate used by the TIBCO Hawkcomponents.

Identity encoding Encoding is applied on the SSL identity - Client's digital certificate.

Ciphers The cipher suite name that can be used.

Debug trace This enables or disables debug tracing while communicating over SSL.

Enable verifyinghost

Select the check box to indicate that the TIBCO Hawk component mustverify the EMS server's certificate.

Enable verifyinghostname

Select the check box to indicate that the TIBCO Hawk component mustverify the name in the CN field of the EMS server's certificate.


141





Adding a Syslog CollectorLogLogic Unity must connect to the syslog server so that a new syslog collector can be used for storingevents.

Procedure





5. From the Type of collector list, select Syslog.


7. Enter the binding host address of the syslog server that connects with LogLogic Unity in theBinding Address field.By default, the binding address is set to 0.0.0.0.

8. Enter the port number of the syslog server that connects with LogLogic Unity in the Binding Portfield.Once you define the port number, you cannot change it for the same collector. You can create a newsyslog collector with a different port number and disable the older collector.

9. Select the protocol from the Protocol list. The options are: UDP and TCP.

10. Select the Source configuration from a list of pre-configured sources.The default value is system. The purpose of source configuration is to help you apply the rightsource type to your incoming data. LogLogic Unity comes with a large number of predefined sourceconfigurations. The source type determines how LogLogic Unity formats your data during parsing.By assigning the correct source type to your data, the data will be parsed appropriately.

11. The Source type field is auto-populated based on your source configuration selection. If the sourceyou have selected has no type ID associated with it, you can enter your own. The valid rangeoptions are from 16,384 to 65,535.

12. Click the Enable Syslog Data Collection check box to enable the collection into the LogLogic Unitysystem.




142


Adding a BusinessWorks Metrics CollectorLogLogic Unity must connect to any existing TIBCO ActiveMatrix BusinessWorks (BW) deployment sothat the agent can publish events generated by the activities BW 6.3.1 to LogLogic Unity. The BWcollector can be used for collecting and storing BW metrics data.

Prerequisites

● TIBCO LogLogic® Unity Plug-in for ActiveMatrix BusinessWorks™ must be installed andconfigured with BW 6.3.1. For installation and configuration information, refer to the TIBCOLogLogic® Unity Plug-in for ActiveMatrix BusinessWorks™ Installation guide.

● BW 6.3.1 must be connected with the LogLogic Unity system.

● Make sure to configure and start the AppSpaces.

Procedure





5. From the Type of collector list, select BusinessWorks.


7. Define the following BusinessWorks filters. If you do not specify any value, all events that match theBW filter will be collected in the LogLogic Unity system.

Field Description

Agent network BW agent network name.

Agent name BW agent name.

Domain BW domain name.

AppSpace BW AppSpace name.

AppNode BW AppNode name.

8. Select the Enable BusinessWorks process activity payload tracking check box to enable processactivity input or output collection into the LogLogic Unity system.

9. If the Enable BusinessWorks process activity payload tracking option is enabled, enter theassociated BW application names in the Application names field. If you do not specify any value,all applications will be considered for collecting BW activity payload data.

10. Select from the Unity domain list of pre-configured domains where events should be stored ordefine a new domain by clicking the Create new domain link. For information on how to create anew domain, see Adding a Domain.

11. Click Save to add a new collector.The newly added collector is displayed on the Collector management page. Once the ActiveMatrixBusinessWorks AppNode connects to LogLogic Unity, by default two source configurations

143


BW_Process and BW_Activity are added. Based on these source configurations, you can now searchfor BW events from the Search page. You can add EQL filters to refine the results.

Editing a CollectorYou can update collectors at any time. You cannot modify the DefaultHawkCollector.

Procedure



3. From the Collector management page, select the collector name that you want to update.The Details panel opens on the right side of the page.

4. Click the Edit link to update the collector information. Make the necessary updates. For BWcollector, you can view connector details.For more information about Hawk collector fields, see Adding a Hawk Collector. For moreinformation about Syslog collector fields, see Adding a Syslog Collector. For more informationabout BW collector fields, see Adding a BusinessWorks Metrics Collector on page 143.

5. Click Save to save the updated information.The updated collector is displayed on the Collector management page.

Enabling or Disabling CollectorsCollectors can be enabled or disabled in order to be activated or deactivated.

Procedure



3. To enable a collector, click the slider in the Enable column to ON.

4. To disable a collector, click the slider in the Enable column to OFF.

Deleting a CollectorYou can delete a collector at any time. However, the source configuration that is created using thiscollector remains in the system. You cannot delete the DefaultHawkCollector.

Procedure



3. From the Collector management page, select the collector that you want to delete by selecting the

check box located next to the Name column and click . To select all collectors in the system,select the check box located next to the Name column header.

144


The Delete button is enabled after you select one or more collectors.

4. In the confirmation window, click Ok to delete the collector and all of its content from the system.The Collector management page is updated immediately.

145


Manage Reports

Using the Reports menu, you can generate reports for monitoring various real-time activities.

Prerequisites

● TIBCO JasperReports® Server: The JasperReports Server builds on JasperReports Library, providesrobust static and interactive reporting, report server, and data analysis capabilities. You mustconfigure the LogLogic Unity system to use a pre-existing JasperReports Server. For instructions, see Configuring a Reports Server.

For more information, refer to the TIBCO JasperReports Server documentation.

Configuring a Reports ServerOnly an admin user can configure the JasperReports Server. All users can run and export reports.

Prerequisites

● TIBCO JasperReports® Server 6.2.0 or TIBCO JasperReports® Server Community Edition 6.2.0.

● Upload the JDBC driver that connects LogLogic Unity with the JasperReports Server. The LogLogicUnity JDBC driver file unity-jdbc-<version>-single.jar is located in the $TIBCO_HOME/logu/<version>/tools/lib directory. For instructions on how to add the driver, refer to the "ManagingJDBC Drivers" section in the TIBCO JasperReports Server Administration Guide.

● On the JasperReports Server, create and save reports that can be accessed from LogLogic Unity.Make sure to enter an SQL statement when creating a report. For instructions on how to create areport, refer to the TIBCO JasperReports Server documentation.

Procedure


2. From the Administration overview landing page, click the Reports link.

3. From the Report management page, click the Configure JasperReports Server link located on themiddle of page or the JasperReports config link located on the upper-right corner of the page, toconfigure the server.

4. In the URL field, enter the JasperReports Server URL that LogLogic Unity system will connect to.The URL format must be as follows: for Professional version: <hostname:port>/jasperserver-pro; forCommunity version: <hostname:port>/jasperserver.For example, localhost:8080/jasperserver-pro or hostname:port/jasperserver

5. Enter the user credentials of the JasperReports Server in the User and Password fields.

6. In the Unity report field, enter the path where LogLogic Unity reports are stored on theJasperReports Server.

7. Click Test to test the configuration.After the successful connection, save the configuration.

8. Click Save to save the configuration.You can now generate any deployed reports.

146


Viewing ReportsAll users can view all the reports, run them, and download and save the generated reports.

On the toolbar, click the Administration icon, the Administration overview landing page displaysdifferent options. Select the Reports link. From the Report management page, you can perform thefollowing tasks:

● Filter reports

You can quickly find the desired report by typing the report name in the Find field. As you starttyping a report name in the Find field, the Report management page is automatically refreshedshowing your selection.

● View reports based on filters

You can use filters to easily find reports in the system. Click the View list to view different filters.

● Sort reports

You can sort any column in ascending or descending order on the Report management page. Clickthe column name or click the arrow (that is displayed on the right side of the column name whenyou click in that column) to sort the column.


You can show or hide columns, except the mandatory column, from the table . Click to view allavailable columns in the table. Select the check box to show the column. Clear the check box to hidethe column from the table. The Report management page is updated immediately.

The Report management page information is described below:

Column Description

Name The name of the report.

Path The path where reports are stored on the JasperReports Server.

Description The description of the report.

Creation date The creation date of the report.

Run Click the button to run the report. For instructions, see Running a Report.

Running a ReportYou can run the deployed reports, regenerate reports with different time ranges, and save generatedreports.

Procedure


2. From the Administration overview landing page, click the Reports link.

147


3. From the Report management page, click Run to generate a report in a PDF file.Based on the selected report, you may need to enter some input fields. If not, the report is generatedand the The report is ready link appears on the Report management page. Click the link to view thereport.

4. If the report requires, enter values for the input fields. Based on the selected report, input fields aredisplayed in the Run and export report window. These may be different for each report type.

In order for a time range picker control to be displayed in the Run and export reportwindow, define "timeRange" as the parameter name and for the time input control, define"time" as the parameter name. In the Time field, specify time as per LogLogic Unity timespecifications, for example, -1d for 1 day. For both fields, the data type should be String.

For example, <parameter name="timeRange" class="java.lang.String"/> or <parametername="time" class="java.lang.String"/>

5. Click Export to run and export a report in a PDF file.Once the report is generated, the The report is ready link appears on the Report management page.Click the link to view the report.

148


Supported Regular Expression Characters

LogLogic Unity supports the following regular expression meta characters, based on Java regularexpressions.

Characters Description

\a Matches ASCII character code 0x07.

\d Matches character in the set "0123456789".

\D Matches any byte not in the set "012345679".

\e The escape character. Matches ASCII character code 0x1b.

\f The form-feed character. Matches ASCII character code 0x0c.

\n The new line (line feed) character. Matches ASCII character code 0x0a.

\r The carriage return character. Matches ASCII character code 0x0d.

\s A white space. Matches white space - \t \n 0x0b \f or \r.

\S A non-white space. Matches any byte not in \s.

\t The tab character. Matches any byte not in 0x09.

\w A word character. Matches any ASCII character in the set underscore,digits, or upper or lower case letter.

\W A non-word character. Matches any bytes not in\w.

\xHH Matches a byte specified by the hex code HH. There must be exactly twocharacters after the \x.

\Q Starts a quoted region. All meta characters lose their meaning until \E. A \\ can be used to put a backlash into the region.

\anytime else Matches the next character.

\k<name> Refers to previous named capture.

[] Specifies a character class - match anything inside the brackets. A leading ^negates the sense of the class - match anything not inside the brackets.Negated character classes are computed from the set of code in the range0-127 - in other words no bytes with the high bit set.

Within a character class the following backslash characters mean the samething as outside the character class: \a, \d, \D, \e, \f, \n, \r, \s, \S, \t, \w,\W, and \xHH.

{num} or {num:num} Specifies a repetition count for the previous regular expression. Num mustbe less than 16. {num} is equivalent to {0:num}.

149


Characters Description

. Matches any byte: 0x00 - 0xFF.

+ Specifies that the previous regular expression is repeated 1 or more times.

* Specifies that the previous regular expression is repeated zero or moretimes.

( ) (?:) Specifies capturing or non-capturing groups.

(?<name>) Specifies capturing named groups.

| Specifies alternation.

? Specifies that the previous regular expression is repeated zero or one time.

anytime else Any other character matches itself.

150


Supported Log Sources

LogLogic Unity supports message body text search for all of the log sources supported by LogLogicLMI and also supports advanced searching of source-specific parsed columns for the following sourcesvia LogLogic Parser.

For details, see the TIBCO LogLogic® Log Source Packages documentation. Note that not all event typessupported by LogLogic LMI may be supported by LogLogic Unity.

Log Source Versions/Platforms Device Category

ADS - Microsoft ActiveDirectory Service

AD Service on Windows 2003Enterprise Edition R2 orWindows 2008 and 2008R2Enterprise Edition

Active Directory

Apache Web Server Apache Web Server (HTTPD)v2.2.4

Apache WebServer

Blue Coat ProxySG Syslog Blue Coat ProxySG SGOS v5.4,v6.1-6.3.0

WebProxy

BMC Remedy Action Request(AR) System

BMC Remedy AR System 7.0on Microsoft Windows 2000 or2003 Server

BMC Remedy ARS

CA SiteMinder - AccessManagement System

eTrust SiteMinder 5.5, 6.0 SP1or SP2 on Windows 2000 withSP4, 2003, or Solaris 8 or 9

Access Control

Check Point Firewall (CPAudit)

4.0 SP8, 4.1 SP6, NGAI R55,NGX R65

Firewall

Cisco ASA Adaptive SecurityAppliance

v7.2, v8.0, and v8.2 - 8.4 UTM

Cisco Content Engine Content Engine with CiscoApplication and ContentNetworking System (ACNS) 4.2or 5.5

Cisco Content Engine

Cisco ESA v7.0, 7.1 Mail Security

Cisco Firewall Services Module(FWSM)

v4.0, v4.1 and v4.1(7) Firewall/VPN

Cisco IOS 12.x, v15.0(M), 15.1(M) Router & Switches

Cisco IPS Cisco IPS 4200 running IPS v6.2or v7.0

IPS

Cisco Identity Services Engine(ISE)

v1.0.2 Access Control

151



Cisco NetFlow Cisco NetFlow v5 or v9, NSEL.IOS XE v15.1(3)M NAT64NetFlow v9

Router

Cisco (Nexus) NX-OS v8.3 Switch

Cisco Secure ACS v4.1, v4.2 and v5.2 Access Control

Cisco VPN3000 - VPN

Cisco Web Security Appliance(WSA)

Async OS v6.3 and v7.1 Web Security

Fortinet (FortiOS) FortiOS 4.0 MR2, v5.0 Firewall

F5 BIGIP Traffic ManagementOperating System (TMOS)

ASM v11.0.0 LTM v11.0.0 Firewall LoadBalancer

GuardiumSQLGuard v6.1 DB IDS/IPS

GuardiumSQLGuard Audit v6.1 DB IDS/IPS

HP NonStop HP NonStop running D48 orlater on a K-series System;G06.20 or later on an S-seriesSystem; H06 or later on anIntegrity NonStop System

System

HP-UX Operating SystemAudit

HP-UX Audit v11iv2 -11i.31 System Audit

IBM AIX Audit v5.3, v6.0, and v6.1. v7.1 System

IBM AIX Operating System v5.3, v6.0, and v6.1. v7.1 System

IBM DB2 Universal Database(UDB)

IBM DB2 UDB v8.1, v8.2, v9.0,v9.5, v9.7 Enterprise ServerEdition on Windows, Solaris,HP-UX, Linux, or AIX

Database

IBM Resource Access ControlFacility (RACF)

SMF record types 80, 81 and 83.RACF on z/OS 1.6, 1.7, 1.8, 1.9,1.10, 1.11-1.13

Access Control

IBM ISS SiteProtector v2.0 Sp5.0,5.1,6.1, 6.2, 8.0 and9.0

IPS

Juniper IDP v5.0 IDS/IPS

Juniper RT_FLOW JunOS v9.3 Firewall

152



Juniper SSL VPN Secure Access v5.5, v6.0 R3, v6.1 R1, v6.2,v6.5, v7.0, v7.1

VPN

Juniper (JunOS) JunOS v9.3, v10.3 & 10.4 UTM

LogLogic Appliance All Platforms System

LogLogic Database SecurityManager

v4.1 Database

McAfee ePolicy Orchestrator ePO v4.0, v4.5, v4.6.0, v4.6.1,v4.6.2; HIPS v7.0, v8.0

IPS

McAfee G2 Sidewinder FW (v6.1, v6.2, v7.x, v8.0-8.3) Firewall/VPN

Microsoft DHCP DHCP Service on Win 2003 or2003 R2 with SP1 or SP3 DHCPService on Win 2008 or 2008 R2with SP1

Microsoft DHCP Application

Microsoft Office SharePointServer

Microsoft Office SharePoint2007, 2010

Content Management

Microsoft Operations Manager MOM 2005 SP1 running onWindows 2003 Server MOM2007 running on Windows2003/2008 Server

System

Microsoft InternetAuthentication Service (IAS)

Microsoft Windows Servers Access Control

Microsoft SQL Server Microsoft SQL Server2005/2008/2012 (ApplicationLogs)

Database

Microsoft SQL Server SQL Server 2005, 2008, 2012Standard or Enterprise

Database

Microsoft Windows Server Windows 2003R2 Server, andWindows Server 2008/R2/2012

Windows Server

Microsoft Windows Server(French)

Windows 2003 Server andWindows 2008R2 Server

Windows Server

Microsoft Windows Server(German)

Windows 2003 Server andWindows 2008 R2 Server

Windows Server

Microsoft Windows Server(Japanese)

Windows 2003 Server andWindows 2008 R2 Server

Windows Server

MySQL Server GDBC v5.5.9 Database

153



NetApp Decru DataFort DataFort FC-series, E- and S-series appliances

Decru Datafort

NetApp Filer NetApp Data ONTAP 7.0,7.3 &8.0 on FAS900, FAS200, F800,GF900, GF800, NearStore R200,150, and 100, and F87. (Notsupported on F700 or F85.)

NetApp Filer, NetApp FilerAudit

Novell eDirectory eDirectory 8.8 on Windows2000 Server with Service Pack4; Windows 2003 ServerEnterprise Edition with ServicePack 1; Windows XPProfessional with Service Pack2; Red Hat Linux AdvancedServer 4; or Novell NetWare 6.5Support Pack 7

LDAP Directory Service

Oracle Database Server Oracle 10g R1/R2 or 11g,10.2.0.4g, 11.2.0.1.0g installed,running on Linux (Fedora Core3), Solaris 9 (64-bit SPARC andIntel i386), HP-UX 11i, or AIX5.3

Database

Other UNIX AIX 5L, HP-UX 11i v2, Solaris8/9/10, RHEL 4/5

System

General Database Collector forOracle

Oracle 11g, 10.2, 10.1, 9.2running on Solaris 9/10

Database

Palo Alto Networks PanOS Palo Alto Networks PanOSv2.1, v3.0, v3.1.0, v4.0.0-4.0.3,v4.1-4.17.v5.0

UTM

RSA ACE/Server ACE/Server 4.x, 5.x, and 6.x onSolaris

Access Control

Reuters KondorPlus All versions Application

Snort v2.4, v2.6, v2.8, v2.9 Intrusion Detection

Sourcefire Sensor v4.1 or v4.6, v4.7 - v4.10 IDS/IPS

Sourcefire Defense Center v4.9.1.7, 4.10.0.0. v5.0.0-5.1.0,5.2.0

IDS/IPS

Squid2 All versions Blue Coat

Sun Solaris Basic SecurityModule (BSM)

Solaris 8/9/10 on Sun SPARC orIntel i386 platforms

Sun Solaris Operating SystemBSM

154



Sybase Adaptive ServerEnterprise (ASE)

Sybase ASE 12.5 or 15.5 on WinXP Pro, Win Svr 2003 Standardor Enterprise; Red HatEnterprise Linux 4; SUSE LinuxEnterprise Server 9; or SunSolaris 8, 9 or 10 (32 or 64-bitSPARC or Intel i386) platforms,v15.7

Database

Symantec SEP v11 and v12 AntiVirus

TIBCO ActiveMatrixAdministrator

V6.3.0 Management Server

TIBCO ActiveMatrix BPM v3.0 Business Process

TIBCO ActiveMatrixBusinessWorks

V5.11 Business Process

TIBCO Administrator V5.7.0 Management Server

TIBCO API Exchange GatewayServer

V2.1 Business Process

TIBCO Hawk Agent V5.11 Business Process

TIBCO Enterprise MessagingService Collector (EMSC)

v6.3.0 EMS

TrendMicro Control Manager v5.0 AntiVirus

TrendMicro OfficeScan v10.0 & v10.5 AntiVirus

Tripwire for Servers Tripwire for Servers 6.5 withTripwire Manager 4.6 runningon Windows 2003 EnterpriseEdition R2

Tripwire Management Station

VMware ESX Server VMware ESX v4.0.0, v4.1.0 orv5.0

Hypervisor

VMware vCenter VMware vCenter Server v4.0.0,v4.1.0, 5.0.0 and v5.1.0

Managment Server

VMware vCenter Orchestrator v4.0.0, 4.1.0, and 4.2.0, v5.1.0 Automation Server

VMware vCloud Director VMware vCenter CloudDirector v1.0.1 through v1.5

Management Server

VMware vShield Edge VMware vShield Server v4.0.0,v4.1.0 or v5.0.0

Firewall

155


Filter Syntax

The following syntactic rules must be followed while constructing filter expressions.

<filterStatement> ::= <conditionList> ;

<conditionList> ::= <andCondition> ( OR <andCondition> )*;<andCondition> ::= <condition> ( AND <condition> )*;<condition> ::= ( NOT )? (<nestedCondition> | <fulltextsearch> | <in> | <between> | <isNull> | <like> | <contains> | <comparison> | <regexp> );

<nestedCondition> ::= “(“ <conditionList> “)”;<fulltextsearch> ::= <String>;<in> ::= <expression> IN “(“ <expressionList> “)” | <expression> IN <timeRange> ;<between> ::= <expression> BETWEEN <expression> AND <expression>;<isNull> ::= <expression> IS ( NOT )? NULL;<like> ::= <expression> ( NOT )? LIKE <expression>;<contains> ::= <expression> ( NOT )? CONTAINS <expression>;<comparison> ::= <expression> <comparator> <expression>;<regexp> ::= <expression> ( NOT )? REGEXP <expression>;<comparator> ::= “=” | “<>” | “!=” | “<” | “<=” | “>” | “>=”;<expression> ::= <value> ( ( “+”|”-“|”*”|”/”|”||” ) <value>)*<value> ::= (<unary>)? <Float> | (<unary>)? <Integer> | <String> | (<unary>)? <columnName> | <function> | (<unary>)? <nestedExpression><unary> ::= “+” | “-“

156


Parsing Rule JSON syntax

The four sample parsing rules in JSON format, for each parser type, are shown below. Make sure tofollow the syntactic rules while constructing a parsing rule in raw mode."parsingRules": [ { "name": "Rule_1", "enabled": "true", "filter": "", "parserProperties": { "separator": ",", "delimiter": "=", "encoding": "UTF-8", "parser_type": "keyvalue", "separatorIsRegex": "true" }, "columns": [ { "name": "col1", "expression": "$col1" } ] }]

"parsingRules": [ { "name": "Rule_2", "enabled": "true", "filter": "", "parserProperties": { "pattern": "\\s", "parser_type": "regex" }, "columns": [ { "name": "col1", "expression": "$col1" } ], } ]

"parsingRules": [ { "name": "Rule_3", "enabled": "true", "filter": "", "parserProperties": { "separator": ",", "escape": "\\", "trim": "true", "parser_type": "columnar", "separatorIsRegex": "true" }, "columns": [ { "name": "col1", "expression": "$col1" } ] } ]

157


"parsingRules": [ { "name": "Rule_4", "enabled": "true", "filter": "", "parserProperties": { "parser_type": "cef", "extensions": [ { "key": "act", "name": "deviceAction", "dataType": "STRING" }, { "key": "app", "name": "applicationProtocol", "dataType": "STRING" }, { ...<predefined extensions> } ], "timeFormats": [ "MMM dd HH:mm:ss", "MMM dd HH:mm:ss.SSS zzz", "MMM dd HH:mm:ss.SSS", "MMM dd HH:mm:ss zzz", "MMM dd yyyy HH:mm:ss", "MMM dd yyyy HH:mm:ss.SSS zzz", "MMM dd yyyy HH:mm:ss.SSS", "MMM dd yyyy HH:mm:ss zzz" ] }, "columns": [] } ]

158


tibco loglogic unity user's guide · 2016-06-02 · tibco loglogic® unity user's guide...

Documents