integrang*splunk*dataand* funcbonality*using*the*splunk ... · the*splunk*rest*api 12!...
TRANSCRIPT
![Page 1: Integrang*Splunk*Dataand* FuncBonality*Using*the*Splunk ... · The*Splunk*REST*API 12! Exposes*an*APImethod*for*every*feature*in*the*product – Run*searches* – Inputdata – Manage*Splunk*configuraons*](https://reader030.vdocuments.mx/reader030/viewer/2022041010/5eb7e3bf047d6e678d213f1d/html5/thumbnails/1.jpg)
Copyright © 2013 Splunk Inc.
Damien Dallimore Developer Evangelist @ Splunk #splunkconf
IntegraBng Splunk Data and FuncBonality Using the Splunk SDK for Java
![Page 2: Integrang*Splunk*Dataand* FuncBonality*Using*the*Splunk ... · The*Splunk*REST*API 12! Exposes*an*APImethod*for*every*feature*in*the*product – Run*searches* – Inputdata – Manage*Splunk*configuraons*](https://reader030.vdocuments.mx/reader030/viewer/2022041010/5eb7e3bf047d6e678d213f1d/html5/thumbnails/2.jpg)
Legal NoBces During the course of this presentaBon, we may make forward-‐looking statements regarding future events or the expected performance of the company. We cauBon you that such statements reflect our current expectaBons and esBmates based on factors currently known to us and that actual events or results could differ materially. For important factors that may cause actual results to differ from those contained in our forward-‐looking statements, please review our filings with the SEC. The forward-‐looking statements made in this presentaBon are being made as of the Bme and date of its live presentaBon. If reviewed aTer its live presentaBon, this presentaBon may not contain current or accurate informaBon. We do not assume any obligaBon to update any forward-‐looking statements we may make. In addiBon, any informaBon about our roadmap outlines our general product direcBon and is subject to change at any Bme without noBce. It is for informaBonal purposes only and shall not, be incorporated into any contract or other commitment. Splunk undertakes no obligaBon either to develop the features or funcBonality described or to include any such feature or funcBonality in a future release.
Splunk, Splunk>, Splunk Storm, Listen to Your Data, SPL and The Engine for Machine Data are trademarks and registered trademarks of Splunk Inc. in the United States and other countries. All other brand names, product names, or trademarks belong to their respecCve
owners.
©2013 Splunk Inc. All rights reserved.
2
![Page 3: Integrang*Splunk*Dataand* FuncBonality*Using*the*Splunk ... · The*Splunk*REST*API 12! Exposes*an*APImethod*for*every*feature*in*the*product – Run*searches* – Inputdata – Manage*Splunk*configuraons*](https://reader030.vdocuments.mx/reader030/viewer/2022041010/5eb7e3bf047d6e678d213f1d/html5/thumbnails/3.jpg)
About Me
![Page 4: Integrang*Splunk*Dataand* FuncBonality*Using*the*Splunk ... · The*Splunk*REST*API 12! Exposes*an*APImethod*for*every*feature*in*the*product – Run*searches* – Inputdata – Manage*Splunk*configuraons*](https://reader030.vdocuments.mx/reader030/viewer/2022041010/5eb7e3bf047d6e678d213f1d/html5/thumbnails/4.jpg)
Developer Evangelist at Splunk Make Talk
4
![Page 5: Integrang*Splunk*Dataand* FuncBonality*Using*the*Splunk ... · The*Splunk*REST*API 12! Exposes*an*APImethod*for*every*feature*in*the*product – Run*searches* – Inputdata – Manage*Splunk*configuraons*](https://reader030.vdocuments.mx/reader030/viewer/2022041010/5eb7e3bf047d6e678d213f1d/html5/thumbnails/5.jpg)
Came from the Splunk Community
5
![Page 6: Integrang*Splunk*Dataand* FuncBonality*Using*the*Splunk ... · The*Splunk*REST*API 12! Exposes*an*APImethod*for*every*feature*in*the*product – Run*searches* – Inputdata – Manage*Splunk*configuraons*](https://reader030.vdocuments.mx/reader030/viewer/2022041010/5eb7e3bf047d6e678d213f1d/html5/thumbnails/6.jpg)
Coder
6
![Page 7: Integrang*Splunk*Dataand* FuncBonality*Using*the*Splunk ... · The*Splunk*REST*API 12! Exposes*an*APImethod*for*every*feature*in*the*product – Run*searches* – Inputdata – Manage*Splunk*configuraons*](https://reader030.vdocuments.mx/reader030/viewer/2022041010/5eb7e3bf047d6e678d213f1d/html5/thumbnails/7.jpg)
From Aotearoa (New Zealand)
7
![Page 8: Integrang*Splunk*Dataand* FuncBonality*Using*the*Splunk ... · The*Splunk*REST*API 12! Exposes*an*APImethod*for*every*feature*in*the*product – Run*searches* – Inputdata – Manage*Splunk*configuraons*](https://reader030.vdocuments.mx/reader030/viewer/2022041010/5eb7e3bf047d6e678d213f1d/html5/thumbnails/8.jpg)
Agenda
![Page 9: Integrang*Splunk*Dataand* FuncBonality*Using*the*Splunk ... · The*Splunk*REST*API 12! Exposes*an*APImethod*for*every*feature*in*the*product – Run*searches* – Inputdata – Manage*Splunk*configuraons*](https://reader030.vdocuments.mx/reader030/viewer/2022041010/5eb7e3bf047d6e678d213f1d/html5/thumbnails/9.jpg)
Agenda
! Developer Pla`orm Overview ! The Basics ! Digging a bit deeper ! IntegraBon Frameworks ! Tools, Libraries and other use cases ! QuesBons (if Bme allowing , else find me aTerwards)
9
![Page 10: Integrang*Splunk*Dataand* FuncBonality*Using*the*Splunk ... · The*Splunk*REST*API 12! Exposes*an*APImethod*for*every*feature*in*the*product – Run*searches* – Inputdata – Manage*Splunk*configuraons*](https://reader030.vdocuments.mx/reader030/viewer/2022041010/5eb7e3bf047d6e678d213f1d/html5/thumbnails/10.jpg)
Developer Pla`orm Overview
![Page 11: Integrang*Splunk*Dataand* FuncBonality*Using*the*Splunk ... · The*Splunk*REST*API 12! Exposes*an*APImethod*for*every*feature*in*the*product – Run*searches* – Inputdata – Manage*Splunk*configuraons*](https://reader030.vdocuments.mx/reader030/viewer/2022041010/5eb7e3bf047d6e678d213f1d/html5/thumbnails/11.jpg)
Powerful Pla`orm for Enterprise Developers
11
REST API
Build Splunk Apps Extend and Integrate Splunk
Simple XML
JavaScript
Django
Web Framework
Java JavaScript Python
Ruby C# PHP
Data Models
Search Extensibility
Modular Inputs
SDKs
![Page 12: Integrang*Splunk*Dataand* FuncBonality*Using*the*Splunk ... · The*Splunk*REST*API 12! Exposes*an*APImethod*for*every*feature*in*the*product – Run*searches* – Inputdata – Manage*Splunk*configuraons*](https://reader030.vdocuments.mx/reader030/viewer/2022041010/5eb7e3bf047d6e678d213f1d/html5/thumbnails/12.jpg)
The Splunk REST API
12
! Exposes an API method for every feature in the product – Run searches – Input data – Manage Splunk configuraBons
! API is RESTful – Endpoints are served by Splunkd – Requests are GET, POST, and DELETE HTTP methods – Responses are Atom XML Feeds or JSON – Search results can be output in CSV/JSON/XML/Raw – AuthenBcaBon is header token based
![Page 13: Integrang*Splunk*Dataand* FuncBonality*Using*the*Splunk ... · The*Splunk*REST*API 12! Exposes*an*APImethod*for*every*feature*in*the*product – Run*searches* – Inputdata – Manage*Splunk*configuraons*](https://reader030.vdocuments.mx/reader030/viewer/2022041010/5eb7e3bf047d6e678d213f1d/html5/thumbnails/13.jpg)
SDKs (SoTware Development Kits)
13
! Our SDKs make it easier to use the REST API – No need to wrangle with lower level HTTP REST semanBcs
! Several different language offerings now available
– Javascript, Java, Python, PHP, C#, Ruby
! Now includes support for more rapid development of Modular Inputs ! Typical Use Cases
– Accelerate your Dev/Test cycles – Integrate with exisBng soluBons – Build new big data applicaBons
![Page 14: Integrang*Splunk*Dataand* FuncBonality*Using*the*Splunk ... · The*Splunk*REST*API 12! Exposes*an*APImethod*for*every*feature*in*the*product – Run*searches* – Inputdata – Manage*Splunk*configuraons*](https://reader030.vdocuments.mx/reader030/viewer/2022041010/5eb7e3bf047d6e678d213f1d/html5/thumbnails/14.jpg)
The Basics
![Page 15: Integrang*Splunk*Dataand* FuncBonality*Using*the*Splunk ... · The*Splunk*REST*API 12! Exposes*an*APImethod*for*every*feature*in*the*product – Run*searches* – Inputdata – Manage*Splunk*configuraons*](https://reader030.vdocuments.mx/reader030/viewer/2022041010/5eb7e3bf047d6e678d213f1d/html5/thumbnails/15.jpg)
Get the Splunk SDK for Java
15
! Current release v1.2 ! Open sourced under the Apache v2.0 license ! Clone from Github : git clone hnps://github.com/splunk/splunk-‐sdk-‐java.git ! Project level support for Eclipse and Intellij IDE’s , or use the development
environment of your choice ! Pre-‐requisites
– JRE 6+ – Splunk installed – Ant
! Maven/Gradle/Ivy repository – hnp://splunk.arBfactoryonline.com/splunk/ext-‐releases-‐local
![Page 16: Integrang*Splunk*Dataand* FuncBonality*Using*the*Splunk ... · The*Splunk*REST*API 12! Exposes*an*APImethod*for*every*feature*in*the*product – Run*searches* – Inputdata – Manage*Splunk*configuraons*](https://reader030.vdocuments.mx/reader030/viewer/2022041010/5eb7e3bf047d6e678d213f1d/html5/thumbnails/16.jpg)
SDK Class Design
16
Service
HTTPService Resource
ResourceCollecBon EnBty
EnBtyCollecBon
Job JobCollecBon
BaseService
Args
JobResultsArgs
![Page 17: Integrang*Splunk*Dataand* FuncBonality*Using*the*Splunk ... · The*Splunk*REST*API 12! Exposes*an*APImethod*for*every*feature*in*the*product – Run*searches* – Inputdata – Manage*Splunk*configuraons*](https://reader030.vdocuments.mx/reader030/viewer/2022041010/5eb7e3bf047d6e678d213f1d/html5/thumbnails/17.jpg)
Code & Demo
! ConnecBng and authenBcaBng ! Hirng a simple REST endpoint ! Simple searches
– Oneshot Blocking Search – Simple Blocking Search
! Indexing an event
17
![Page 18: Integrang*Splunk*Dataand* FuncBonality*Using*the*Splunk ... · The*Splunk*REST*API 12! Exposes*an*APImethod*for*every*feature*in*the*product – Run*searches* – Inputdata – Manage*Splunk*configuraons*](https://reader030.vdocuments.mx/reader030/viewer/2022041010/5eb7e3bf047d6e678d213f1d/html5/thumbnails/18.jpg)
Digging a Bit Deeper
![Page 19: Integrang*Splunk*Dataand* FuncBonality*Using*the*Splunk ... · The*Splunk*REST*API 12! Exposes*an*APImethod*for*every*feature*in*the*product – Run*searches* – Inputdata – Manage*Splunk*configuraons*](https://reader030.vdocuments.mx/reader030/viewer/2022041010/5eb7e3bf047d6e678d213f1d/html5/thumbnails/19.jpg)
Code & Demo ! Searching (also demonstrates handling & displaying results )
– Asynchronous with Job object – RealBme Search – ExporBng – Paging through large result sets – Saved searches and dynamic dispatch arguments
! Indexing data – TCP – UDP – Index oneshot upload
! Managing Splunk (also demonstrates state management) – create a Role enBty – create a User enBty
! Namespaces
19
![Page 20: Integrang*Splunk*Dataand* FuncBonality*Using*the*Splunk ... · The*Splunk*REST*API 12! Exposes*an*APImethod*for*every*feature*in*the*product – Run*searches* – Inputdata – Manage*Splunk*configuraons*](https://reader030.vdocuments.mx/reader030/viewer/2022041010/5eb7e3bf047d6e678d213f1d/html5/thumbnails/20.jpg)
IntegraBon Frameworks
![Page 21: Integrang*Splunk*Dataand* FuncBonality*Using*the*Splunk ... · The*Splunk*REST*API 12! Exposes*an*APImethod*for*every*feature*in*the*product – Run*searches* – Inputdata – Manage*Splunk*configuraons*](https://reader030.vdocuments.mx/reader030/viewer/2022041010/5eb7e3bf047d6e678d213f1d/html5/thumbnails/21.jpg)
Let’s Make Life Even Easier
! Splunk SDK for Java makes using the REST API easier ! What can we do to make the SDK easier to use ? ! Integrate into standard enterprise Java soTware frameworks
– Spring IntegraBon (code & demo) – Apache Camel (code & demo) – Mule ESB
21
![Page 22: Integrang*Splunk*Dataand* FuncBonality*Using*the*Splunk ... · The*Splunk*REST*API 12! Exposes*an*APImethod*for*every*feature*in*the*product – Run*searches* – Inputdata – Manage*Splunk*configuraons*](https://reader030.vdocuments.mx/reader030/viewer/2022041010/5eb7e3bf047d6e678d213f1d/html5/thumbnails/22.jpg)
Tools, Libraries & Other Use Cases
![Page 23: Integrang*Splunk*Dataand* FuncBonality*Using*the*Splunk ... · The*Splunk*REST*API 12! Exposes*an*APImethod*for*every*feature*in*the*product – Run*searches* – Inputdata – Manage*Splunk*configuraons*](https://reader030.vdocuments.mx/reader030/viewer/2022041010/5eb7e3bf047d6e678d213f1d/html5/thumbnails/23.jpg)
Splunk Java Logging
23
! A logging framework to allow developers to as seamlessly as possible integrate Splunk best pracBce logging semanBcs into their code
! Transport log events to Splunk directly from your code ! Custom handler/appender implementaBons(REST and Raw TCP) for common Java logging frameworks ̶ LogBack ̶ Log4j ̶ java.uBl.logging
! UBlity classes for formarng log events ! Configurable in memory buffer to handle network outages
![Page 24: Integrang*Splunk*Dataand* FuncBonality*Using*the*Splunk ... · The*Splunk*REST*API 12! Exposes*an*APImethod*for*every*feature*in*the*product – Run*searches* – Inputdata – Manage*Splunk*configuraons*](https://reader030.vdocuments.mx/reader030/viewer/2022041010/5eb7e3bf047d6e678d213f1d/html5/thumbnails/24.jpg)
Eclipse Plugin
24
! Adds extensions to the Eclipse soTware development pla`orm to simplify creaBng projects using the Splunk SDK for Java
! A new "Splunk SDK for Java" project type ! A set of code templates for common tasks using the Splunk SDK for Java, plus opBonal data parsing and logging capabiliBes
! A launcher for Java applicaBons that logs JVM acBvity to a Splunk instance – JMX java.lang MBeans – Byte Code InjecBon method level tracing
![Page 25: Integrang*Splunk*Dataand* FuncBonality*Using*the*Splunk ... · The*Splunk*REST*API 12! Exposes*an*APImethod*for*every*feature*in*the*product – Run*searches* – Inputdata – Manage*Splunk*configuraons*](https://reader030.vdocuments.mx/reader030/viewer/2022041010/5eb7e3bf047d6e678d213f1d/html5/thumbnails/25.jpg)
Other JVM Languages
Scala Groovy Clojure
Javascript(Rhino) JRuby PHP(Quercus)
Ceylon Kotlin Jython
We don’t need SDK’s for these languages , we can just use the Java SDK !
25
![Page 26: Integrang*Splunk*Dataand* FuncBonality*Using*the*Splunk ... · The*Splunk*REST*API 12! Exposes*an*APImethod*for*every*feature*in*the*product – Run*searches* – Inputdata – Manage*Splunk*configuraons*](https://reader030.vdocuments.mx/reader030/viewer/2022041010/5eb7e3bf047d6e678d213f1d/html5/thumbnails/26.jpg)
HUNK (Splunk AnalyBcs for Hadoop)
! A new product offering from Splunk ! Allows you to use the power and simplicity of Splunk to search over data locked away in HDFS
! Sits on top of HDFS as if it was a naBve Splunk Index ! Virtual Indexes ! So you can use the Splunk SDK for Java to develop your Hadoop applicaBons in exactly the same way that you currently use it.
26
![Page 27: Integrang*Splunk*Dataand* FuncBonality*Using*the*Splunk ... · The*Splunk*REST*API 12! Exposes*an*APImethod*for*every*feature*in*the*product – Run*searches* – Inputdata – Manage*Splunk*configuraons*](https://reader030.vdocuments.mx/reader030/viewer/2022041010/5eb7e3bf047d6e678d213f1d/html5/thumbnails/27.jpg)
Housekeeping
![Page 28: Integrang*Splunk*Dataand* FuncBonality*Using*the*Splunk ... · The*Splunk*REST*API 12! Exposes*an*APImethod*for*every*feature*in*the*product – Run*searches* – Inputdata – Manage*Splunk*configuraons*](https://reader030.vdocuments.mx/reader030/viewer/2022041010/5eb7e3bf047d6e678d213f1d/html5/thumbnails/28.jpg)
Where to Go for More Info Twiner @splunkdev Blog hnp://blogs.splunk.com/dev Demos hnp://demos.splunk.com
Email [email protected] Portal hnp://dev.splunk.com Github hnps://github.com/splunk
28
![Page 29: Integrang*Splunk*Dataand* FuncBonality*Using*the*Splunk ... · The*Splunk*REST*API 12! Exposes*an*APImethod*for*every*feature*in*the*product – Run*searches* – Inputdata – Manage*Splunk*configuraons*](https://reader030.vdocuments.mx/reader030/viewer/2022041010/5eb7e3bf047d6e678d213f1d/html5/thumbnails/29.jpg)
Links Github Gists for code examples in this presentaBon : hnps://gist.github.com/damiendallimore SDK docs at dev.splunk.com : hnp://dev.splunk.com/view/SP-‐CAAAECN Splunk SDK for Java Github repository : hnps://github.com/splunk/splunk-‐sdk-‐java Splunk Spring IntegraBon repository on Github : hnps://github.com/SpringSource/spring-‐integraBon-‐extensions/tree/master/spring-‐integraBon-‐splunk Splunk Spring IntegraBon demo on Github : hnps://github.com/damiendallimore/spring-‐integraBon-‐splunk-‐webex-‐demo
Splunk Apache Camel repository on Github : hnps://github.com/pax95/camel-‐splunk Splunk Apache Camel demo on Github : hnps://github.com/pax95/camel-‐splunk-‐example Splunk Eclipse plugin : hnp://dev.splunk.com/view/splunk-‐plugin-‐eclipse/SP-‐CAAAEQP Splunk Java Logging on Github : hnps://github.com/splunk/splunk-‐library-‐javalogging
Splunk Java Agent on Github : hnps://github.com/damiendallimore/SplunkJavaAgent Splunk Android SDK on Github : hnps://github.com/damiendallimore/splunk-‐sdk-‐android Splunk REST API reference : hnp://docs.splunk.com/DocumentaBon/Splunk/latest/RESTAPI/RESTcontents Free Splunk download : hnp://www.splunk.com/get?r=header
Best pracBce logging overview : hnp://dev.splunk.com/view/logging-‐best-‐pracBces/SP-‐CAAADP6 Splunk SDK for Java videos : hnp://dev.splunk.com/view/get-‐started/SP-‐CAAAECH HUNK Beta video : hnp://www.splunk.com/view/SP-‐CAAAH2F Maven/Gradle/Ivy Repository : hnp://splunk.arBfactoryonline.com/splunk/ext-‐releases-‐local
29
![Page 30: Integrang*Splunk*Dataand* FuncBonality*Using*the*Splunk ... · The*Splunk*REST*API 12! Exposes*an*APImethod*for*every*feature*in*the*product – Run*searches* – Inputdata – Manage*Splunk*configuraons*](https://reader030.vdocuments.mx/reader030/viewer/2022041010/5eb7e3bf047d6e678d213f1d/html5/thumbnails/30.jpg)
Contact me
Email : [email protected] Twi6er : @damiendallimore Skype : damien.dallimore Github : damiendallimore Splunkbase : damiend Slideshare : hnp://www.slideshare.net/damiendallimore
I will be around all week , feel free to grab me for a chat or a beer or both J
30
![Page 31: Integrang*Splunk*Dataand* FuncBonality*Using*the*Splunk ... · The*Splunk*REST*API 12! Exposes*an*APImethod*for*every*feature*in*the*product – Run*searches* – Inputdata – Manage*Splunk*configuraons*](https://reader030.vdocuments.mx/reader030/viewer/2022041010/5eb7e3bf047d6e678d213f1d/html5/thumbnails/31.jpg)
Thank you !