splunk 6.2 new features

cleverdata.ru | [email protected]

Ge#ng value from your data just got really fast. New features SPLUNK> 6.2

Dmitry Anoshin, Solu>ons Architect


New Approach in Analy?cs

Collect Data Analyse Prepare Data (clean, transform)

Collect Data

Tradi>onal Approach Tradi>onally, a lot of >me has been spent collec>ng and preparing data. Eventually you get to ask the ques>ons of the data, start to create the right analy>cs and get the insight you need from it. This can take a lot of ?me.

Fresh Approach We need to get the value from the data as quickly as possible, in order find valuable insights.

Prepare Data Analyse

With Splunk Enterprise 6.2 we’ve focused on three main areas of func>onality: 1)  Easier data on-‐boarding, data prepara>on and Advanced Field Extractor (AFX) 2)  More Powerful Analy>cs for Everyone – Instant Pivot, Event PaWern Detec>on and Prebuilt Panels 3)  Simplified Management at Scale – Search Head Clustering and Distributed Management Console


Key Features • Data Explora>on • Knowledge Management • Dashboard Enhancements • Mission-‐Cri>cal Enterprise


Data Explora>on


Home Page Redesign

How it works •  Users can navigate, and choose an exis>ng dashboard from

within the home page itself •  Alterna>vely, users can mkae any dashboard their home page

from the dashboard lis>ng page, or within the dashboard itself •  This selec>on gets saved to the user's user-‐prefs.conf

Splunk 6.2 introduces a new home page to provide users quicker access to relevant data, allowing users to select any exis>ng dashboard to use as their custom home page.


Event PaWern Detec>on


Bubble Chart

Use bubble charts for mul>-‐dimensional numeric analy>cs. Bubble chart specific op>ons: •  Minimum bubble size <op>on name="char>ng.chart.bubbleMinimumSize">10</

op>on> •  Maximum bubble size <op>on name="char>ng.chart.bubbleMaximumSize">50</

op>on> •  Bubble size by area or radius <op>on name="char>ng.chart.bubbleSizeBy">area</

op>on>


Instant Pivot Enables you to open any query in the Pivot interface, without requiring the crea>on of a data model. This means that you have the flexibility to choose what interface to explore your data. How it works •  Ephemeral data model is created that collects user specified fields within Pivot as a single,

flat object •  User can save their Pivot (addi>onally prompts user to save data model)


Ultra Drilldown

Provides more explicit drilldown controls, without requiring users to know any of the hidden modifier keys.


Knowledge Management


Consolidated Workflow

We’ve made it much easier to find your way to the appropriate input configura>on. Instead of selec>ng from a confusing list of sources, start with a simple choice of “upload, monitor, or forward” and you’ll find yourself in a simple wizard-‐style workflow of defining the appropriate parameters for the data you want to add.


Splunk can index any machine data. Common data sources are:

Data Sources


With Forwarder Inputs, you are able to push input configura>ons to Splunk instances configured as deployment clients. Simply select one or more forwarders, provide a group name, and you’ll be able to create data inputs on them in the same way you create inputs through the UI on your indexers.

Forwarder Inputs


New Data Preview

The new data preview u>lity will make it easier for you to create the right sourcetype for your data. In the advanced sec>on, you’ll be able to choose a charset from a list, and see how changes you make to your sourcetype are reflected in props.conf.


In order to beWer choose the appropriate sourcetype when adding data we built a new sourcetype picker, which includes categories and descrip>ons. To add categories and descrip>ons to your sourcetypes, modify the sourcetype_metadata.conf file. [docs link]

Sourcetype categories and descrip?ons


Advanced Field Extractor

In Splunk 6.2, it easier to extract fields from your data with the Advanced Field Extractor (AFX). A replacement of the exis>ng field extrac>on u>lity, AFX enables you to easily capture mul>ple fields in a single extrac>on and specify required text to filter events for extrac>on (improving accuracy and efficiency). AFX also provides a number of methods for detec>ng false posi>ves in order to help you validate your field extrac>ons and improve the accuracy of your fields.


With the search filter, you can enter a keyword to filter events by, so that you can beWer drill down to the exact event containing your desired value to extract into a field. This can also help you validate extrac>ons – once you’ve selected a sample value to extract, you can type in other values that should be extracted to this field in the search filter to make sure they are also being extracted (extrac>ons are highlighted in the event lis>ng).

Search filter


Sample Events & Highlight-‐to-‐Extract

In the legacy field extractor, Splunk Enterprise asks you to type in sample values for fields to extract. In the (new) Advanced Field Extractor, you’re now asked to select a sample event from the event lis>ng to extract values from, and in the next step, highlight values in the chosen sample event to create field extrac>ons.


Sample Events & Highlight-‐to-‐Extract Cont.

You can select addi>onal sample events in the “Select Sample” and “Select Fields” steps of the workflow, and highlight field values (in the “Select Fields” step) from the addi>onal sample events to improve the accuracy of the extrac>on that is generated.


Rather than extract one field at a >me, try highligh>ng mul>ple values in your selected sample event – you’ll see color-‐coded highligh>ng apply to the listed events. To remove or rename a field, simply click its value in the sample event.

Extract mul?ple fields at once


Some>mes a source type may contain different kinds of events, and in order to extract fields from the various paWerns you may want to go through the extrac>on process more than once, for each of the paWerns. Or maybe you want to make sure that a value is only extracted to a field from specific events. To improve both accuracy and efficiency, you can now specify required text: values in the event will be extracted as fields only when the event contains the required text. At this >me, only one value can be defined as required text, and you cannot require and extract the same value. To require text, highlight as if to extract, but choose “require”.

Specify Required Text


Aqer defining field extrac>ons in the “Select fields” step, you should verify their efficacy before saving them. In the “Validate” step, the Advanced Field Extractor offers a variety of methods for iden>fying values that shouldn’t have been extracted (false posi>ves) and lets you provide feedback for the purpose of improving accuracy.

Valida?ng and fixing field extrac?ons


To beWer inspect the extracted values for a field, click on its field tab. You will see a chart showing its values (similar to “ | top <field>” or “ | stats count by <field>”). If you see an incorrect value, you can click it to add it as a filter. When you click a field value in a field stats tab, the filter box will fill with a “field=value” search and the events you see listed will be limited to those that match the filter.

Field Stats


To see what fields have been previously defined for the chosen sourcetype, click the “Exis>ng Fields” buWon at the top right. You’ll see a list of each field that’s defined, and a link to open it in manual mode (of the Advanced Field Extractor). Note: Only fields defined through the field extractor, or as EXTRACT-‐ groups in props.conf, will be listed in this sec>on.

Exis?ng Fields


Manual Mode

If you speak Regular Expression, you may define one manually rather than highligh>ng values to extract. Currently only accessable in the “Select sample” step (this will change), manual mode will bypass the wizard workflow and you will not be able to provide counterexample feedback to improve your RegEx. You will however, get to take advantage of the event lis>ng features like filtering on keywords and match/non-‐match, event lis>ng op>ons, field stats, and highligh>ng of extracted values.


App Key Value Store

With Splunk 6.2, App developers can now make use of the App Key Value Store to persist data associated with their applica>ons. The common uses for the K/V Store are to both enrich machine data within Splunk Apps and a variety of other scenarios which require persistence of Splunk App specific informa>on. •  Highly flexible, secure and scalable storage and retrieval capabili>es. •  Ability to create, update and query data collec>ons using a rich set of REST APIs. •  Ability to use the exis>ng Splunk lookup commands (inputlookup, etc.) against data in

the App value key store. •  Enforce Splunk’s Role-‐based access control to data in the App key value store. •  Monitoring and management dashboards to understand the performance

characteris>cs of the App key value store.


Dashboard Enhancements


Build custom dashboards faster, leveraging pre-‐built dashboard panels packaged within apps. Enables anyone to build their own custom dashboard, leveraging pre-‐built dashboard panels packaged within apps. Panels allow app developers to create and package re-‐usable dashboard building blocks that enable end-‐users to leverage in adap>ng applica>ons to meet their specific requirements and use cases. •  Improve developer efficiency to support more use cases, requirements, and target users without addi>onal effort •  Easily adapt applica>ons to meet the specific needs of each customer and organiza>on •  Expand engagement across mul>ple groups and levels of users with a single applica>on •  Enable any user to create personalized dashboards with informa>on rela>ve to their specific use case •  Improves user efficiency with enhanced content browsing, search, and discovery Key Features •  New add content workflow for inline panels, pre-‐built panels, reports, and cloning content from other dashboards •  Improved add inline content flow, including the ability to select viz type directly within the add func>on •  Ability to search across all available content •  Preview content before adding to the dashboard •  Convert to Prebuilt Panel to maximize reusability •  Convert panel to inline to further customize

Panels


Mul?-‐Search Management

Configure mul>ple background searches, with explicit post process references throughout your dashboard. This allows users to: •  Op>mize performance of the page by execu>ng a single search that mul>ple visualiza>ons leverage •  Note that there is a limit of 10k results passed down to post process search (for event-‐based searches) •  Drive mul>ple related form inputs from a single global search, and post process within each input •  In some cases, perform token-‐based searches within a post process so that you don't have to execute an expensive global search

every >me a new input is selected How it Works •  Global searches may be instan>ated anywhere on the page (including within panels) •  Global searches (any search that you want to later perform a post process against) must include an "id" <search

id="global_search">...</search> •  Explicitly bind a postprocess query to any search using a base="" <search id="post_process_1" base="global_search">...</search> •  Included a saved search within a dashboard now uses the syntax ref="" <search ref="mySavedSearch">...</search> •  New search syntax uses new syntax for earliest and latest (NOT earliestTime or latestTime)


Set mul>ple tokens within form inputs to drive mul>ple searches, beWer labeling, and more. Key use cases include: •  Set tokens for both label and value to use throughout your dashboard •  Use this to create a special empty/null choice that includes a unique token transforma>on •  Use the selec>on of a given form input to unset other tokens on the page •  Create a "simple" >me range picker dropdown input that sets a unique earliest and latest token •  Set mul>ple tokens based on search results

Input Mul?-‐token SeVer


Panel Title Support

Organize dashboard content into panels, and set a >tle to describe the content. Panels are fully supported dashboard objects, which includes •  Panel >tles (as well as element >tles) can be edited now in-‐place •  Dele>ng a panel will delete all sub elements •  Convert to pre-‐built panel will save as a resusable panel object, and convert to

reference


Free Form Input Support within Dropdown and Mul?-‐select

Enable users to enter free form text within dropdown and mul>-‐select inputs. This allows users to •  Enter free form value that includes wildcards, like "*Risk" •  Useful for host selec>ons, where users can enter par>al domains, par>al hostnames, and

more

This feature is controls via the op>on, <allowCustomValues>


Full Object ID Support

<form> <label>sample id example</label> <descrip>on></descrip>on> <fieldset submitBuWon="false"> <input type=">me" token=">me" id=">me_picker"> <label></label> <default> <earliestTime>-‐60m@m</earliestTime> <latestTime>now</latestTime> </default> </input> </fieldset> <row id="row_1"> <panel id="panel_1"> <>tle>Top Sourcetype</>tle> <table id="table_1"> <search> <query>index=_internal | top sourcetype</query> <earliest>$>me.earliest$</earliest> <latest>$>me.latest$</latest> </search> <op>on name="wrap">true</op>on> <op>on name="rowNumbers">false</op>on> <op>on name="dataOverlayMode">none</op>on> <op>on name="drilldown">cell</op>on> <op>on name="count">10</op>on> </table> </panel> </row> </form>

Assign unique IDs to rows, panels, elements, and form inputs for easier custom js/css handling.


Mission-‐Cri>cal Enterprise


Search Head Clustering provides high availability by replica>ng the user configura>on se#ngs, dashbaords, and reports across search heads. Users can use any member of the clusters and they will get the same user experience

Search Head Clustering

Key features: •  Automa>c synchroniza>on of files

between dashboards •  Search heads can be added /

removed any>me without bringing down the en>re cluster

•  Easier to administer the cluster through centralized interface

•  Cost effec>ve, no need to buy expensive shared storage system

•  No single point failure in the architecture


The new bucket status page makes it easier to monitor large scale clusters. It provides an in-‐depth view of all cluster related recovery opera>ons. If the cluster needs any fix-‐ups, then the new bucket status page provides an accurate view of all the fix-‐ups ac>vi>es. It shows the buckets that are fixed to meet replica>on policies vs search policies. The status page can be viewed at an aggregate level for all indexes or for specific index for detailed analysis. The page also shows any excess buckets, if present, in the cluster. The excess buckets can be removed through the bucket status page itself.

Indexer Cluster Monitoring


Monitor key usage and performance metrics across the en>re Splunk topology The feature builds upon plaworm instrumenta>on and other features added in the 6.1 release to enhance the Splunk Admin's awareness of their distributed Splunk Topology and includes Splunk Dashboards/Views that report on three key areas: •  Search Usage and Performance at Deployment-‐wide and Individual levels •  Indexing Usage and Performance at Deployment-‐wide and Individual levels •  Plaworm Resource U>liza>on (CPU/Memory/Disk) at Deployment-‐wide and Individual

levels The feature also includes several "Plaworm Alerts" that allow the Splunk Admin to enable email alerts for pre-‐packaged condi>ons that may be detrimental to the opera>on of Splunk

Distributed Monitoring Console


Thank you!

splunk 6.2 new features

Technology