an introduction to logger and esm web services apiswhat are web services? wikipedia definition •...

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

An introduction to Logger and ESM Web Services APIs Shivdev Kalambi Principal developer and manager correlation team

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 2

Agenda

Logger Web Services APIs • Look at the Login Service, Report Service and Search Service • Look at some use cases ESM Web Services APIs • Look at the Login Service, Query Viewer Service, and Report Service • REST & SOAP Examples Q&A


Logger Web Services APIs


Web Service APIs What are web services?

Wikipedia definition • Web services are typically

application programming interfaces (API) or web APIs that can be accessed over a network, such as the Internet, and executed on a remote system hosting the requested services.

External client

Internal client

Server that exposes Web Services (e.g. Logger)

Web


So what are we doing here?

The idea is simple Fetch data from Logger and apply it to your use case

Do some analytics? Show the source Address on a map

Write a simple client? For the command line folks


What must I know?

Got data?

Prerequisites • Logger Search Syntax • Logger Reporting (SQL) • Knowhow of Web Services in general

• Write code for simple SOAP clients

100110011010101001101001101010 ?


Logger Web Service APIs APIs available in Logger v5.3

APIs Available • Services

• LoginService • ReportService • SearchService

• WSDL Location • https://<LoggerHost><:Port>/soap/services/<ServiceName>/<ServiceName>.wsdl • https://192.168.35.9/soap/services/ReportService/ReportService.wsdl

https://192.168.35.9/soap/services/ReportService/ReportService.wsdl


Methods Arguments Return Description

login username, password,timeout cookie Login to a logger and establishes a cookie

logout cookie Ends a session identified by the cookie

getVersion String Returns the version of the web service

extendSession cookie Extends the session identified by the specified cookie

LoginService

Example String cookie = loginService.login(“admin”, “password”, 120);



getReportGroups cookie Group[] Get the Report Groups (Categories)

getReportsInGroup groupID, cookie Report[] Get the Reports in a Group

runReport

reportId, startTime, endTime, scanLimit, rowLimit, devices, deviceGroups, storageGroups, reportParams, reportFormat, cookie

String Base64 encoded Report Result (for eg. in CSV format).

getDevices cookie String[] Get a list of devices

ReportService

Example String report = reportService.runReport(report_ID, System.currentTimeMillis() – 2 * 60 * 60 * 1000, System.currentTimeMillis(), 10000, 100, null, null, null, null, “csv”, cookie);



startSearch queryString, startTime, endTime, cookie

Starts the search

endSearch cookie Ends a search session identified by the cookie

getNextTuples count, timeout, cookie Tuple[] Get an array of Tuples that matched the Search

hasMoreTuples cookie boolean Find out whether the search has more tuples

getHeader cookie String[] The Format of the Data in the Tuples

SearchService


Sample Java Code Simple Search using the Iterator Pattern // Login String cookie = loginService.login("admin", "password", 60); // Start the Search searchService.startSearch("ERROR“, System.currentTimeMillis() - 10 * 60 * 1000, System.currentTimeMillis(), cookie); // Loop over and find resulting rows while (searchService.hasMoreTuples(cookie)) { Tuple [] tuples = searchService.getNextTuples(500, 1000, cookie); if (tuples != null) { for (Tuple tuple : tuples) { String [] arr = tuple.getData(); // Custom Processing of the data } } } // End the Search searchService.endSearch(cookie); loginService.logout(cookie);


Use case 1

Command Line Utility • Run a search (or report) from the Logger

Web UI – Logger | cef name | top 5 name

• Simulate the same search using Web Services from a command line utility


Use case 1 Command line utility


Use case 1

ESM/Logger Integration • When a rule fires in ESM • Execute the utility to query logger for

events • Send an email with events

INtegrate the command line utility with ESM


Use case 2

Run a search CEF | cef sourceAddress | top 5

sourceAddress Run a report using this SQL

SELECT arc_sourceAddress, COUNT(arc_eventId) FROM events GROUP BY arc_sourceAddress ORDER BY COUNT (arc_eventID) DESC LIMIT 5

Feed the results to a mapping tool

Plot geo locations for top 5 source IP addresses on a map


Best practices

Some points to keep in mind • Login Service

– Always logout – Sessions get purged but will take a while to clean up • Search Service

– Always endSearch – The Searcher will be instantly cleaned up – From v5.3, you will see performance improvements in the getNextTuples(count, timeout, cookie) – by

changing the count • Report Service

– Data returned from runReport(…) call is Base64 encoded – so you would need to decode it


Writing clients

Please note • Users are expected to write their own SOAP Clients • We provide a reference JAVA implementation and have provided some sample JAVA code to serve as an

example • Documentation is available on the customer support site


ESM Web Services APIs


ESM Web Services APIs

Introduction • Login Service • Query Viewer Service

– Fetching data using REST – Fetching data using SOAP

• Java Code Snippets • Report Service • Required Libraries & Interesting Observations


Use cases

Fetch query viewer data • Sample Query Viewer • Fetch data

– Using REST – Using SOAP

Top 10 most common events


Use case 1 REST call to login service

https://localhost:8443/www/core-service/rest/LoginService/login?login=admin&password=password

https://localhost:8443/www/manager-service/rest/QueryViewerService/getMatrixData?authToken=__&id=__

Test the REST • From the browser • Invoke LoginService

– Copy the authToken • Invoke QueryViewerService

– Pass the QueryViewer ID


Use case 1 REST call to QueryViewer service

https://localhost:8443/www/manager-service/rest/QueryViewerService/getMatrixData?authToken=__&id=__

QueryViewerService using REST

QueryViewerService using SOAP


Use case 2

SOAP Call to login service // Set the Base URL System.setProperty("com.arcsight.coma.client.ws.baseURL", "https://” + host + "/www/"); // localhost:8443 // Get the LoginService and login LoginServiceClientFactory factory = new LoginServiceClientFactory(); LoginService service = factory.createClient(); String authToken = service.login(null, “admin”, “password”); // This authToken is required in subsequent calls


Use case 2

SOAP call to query viewer service // Get the QueryViewerService and get the data QueryViewerServiceClientFactory factory = new QueryViewerServiceClientFactory (); QueryViewerService service = factory.createClient(); MatrixData md = service.getMatrixData(authToken, "cwswTlzgBABCKipuKVcyzlg=="); // Get the Column Names List<String> headers = md.getColumnHeaders(); int col = 0; for (String header : headers) { System.out.printf((col++%2 == 0 ? "%60s" : "%20s\n"), header); } // Get the Data List<ListWrapper> rows = md.getRows(); for (ListWrapper row : rows) { List value = row.getValue(); for (Object obj : value) { System.out.printf((col++%2 == 0 ? "%60s" : "%20s\n"), obj); } }


Use case 2

I found that I needed the following static block to trust the hostname // Static Block static { HttpsURLConnection.setDefaultHostnameVerifier(new HostnameVerifier() { public boolean verify(String hostname, SSLSession session) { // Make sure that hostname is valid return true; } }); }


Use Case 3 REST Call to Report Service

Start the Report Generation

https://localhost:8443/www/manager-service/rest/ArchiveReportService/initDefaultArchiveReportDownloadById?authToken=_&reportId=_&reportType=Manual

Get the Download ID and download the report

https://localhost:8443/www/manager-service/fileservlet?file.command=download&file.id=DOWNID










Required libraries and interesting observations

Tips from an end user • Even though it’s SOAP under the covers AXIS2 libraries didn’t work

– manager-ws-client-1.2.0.release.107.jar – core-ws-client-1.5.0.release.51.jar – coma-infrastructure-1.4.0.release.240.jar

• For now, the SOAP APIs can only be written in Java and using these libraries • I was not able to get it to work with AXIS2 libraries in the CLASSPATH

– Marshalling Errors • Don’t forget to implement a HostnameVerifier (by default it will NOT be a verified hostname) • Documentation is available on the Support Site


Recap Key Takeaways • Logger Web Services APIs

– Supports only SOAP – Login Service – Local Authentication – Search Service – Start Search and Iterator Pattern to go over resulting Tuples – Report Service – Run a Report and get back a Base64 encoded result

• ESM Web Services APIs – REST – for simple use cases – SOAP – For now, Java clients using the provided libraries – GWT-RPC is also used by our UI team


After the event Meet these folks

Find out more

Attend these sessions

• Session ID: 1257 Title: Gain quick and relevant solutions made possible by the HP ArcSight Logger API.

• Logger PMs: Roopak Patel & Suresh Venkatraman

• ESM PMs: Monica Jain & Saran Selvaraj

• ESM, CORRE, Logger Dev: Anurag Singla, David Wiser, & Vivek Vallachira

• Contact your sales rep

• Visit the Social Pages at: http://www.facebook.com/HPSecure Twitter: @HPSecure

• Download the slides at: http://protect724.arcsight.com/

Your feedback is important to us. Please take a few minutes to complete the session survey.

http://www.facebook.com/HPSecure

http://www.facebook.com/HPSecure


Thank you


Security for the new reality

an introduction to logger and esm web services apiswhat are web services? wikipedia definition •...

Documents