KrakenIntroduction to theOSGi based security platform

Introduction

Author– xeraph@nchovy.com

Target Audience– Information security solution developer

– Java developer who is interest on OSGi application stack

Introduction

Security Platform– Most security solutions shares common requirements

• Aggregate some informations from agents or sensors

– System status (performance, software versions, etc)

– Log from various data store (file, database, syslog, snmp trap, etc)

• Inspect informations and applies security policies

• Trigger configured actions

– Send event or alarm through various media (e.g. sms, email)

– Control other security solutions (e.g. firewall policy)

– Drop packet

• Generate statistics and reports

– Security Platform provides

• common functionalities as components

• reliable and solid development framework

Introduction

Common Problems– Lack of pipeline architecture style

• Security solutions need pipelined message processing

– IDS message flow

» Generate intrusion event

» Apply response policy to event

» Respond to an intrusion (syslog, trap, sms, RST packet, etc)

– ESM message flow

» Read logs from file, database, network, and so on.

» Parse logs and normalize them

» Filter logs by policy

» Send event to realtime view of console

» Summarize logs and generate reports

Introduction

Common Problems– Lack of pipeline architecture style

• Solutions that did not consider pipelining encounter following problems:

– Hard wiring of message path

» You should modify caller side if you want to add new feature

– Hard to provide optional feature and remove unused feature

» You should maintain many branches

– Long downtime

» You should kill the process, patch some files, and start it

– Lack of troubleshooting support

• Do you think logging is sufficient?

– Error log does not provide detail information

– It’s not that easy to trace a tons of debug logs

– Add debug log to code and wait until problem appears again?

Introduction

Common Problems– Lack of configurability

• Security appliances should provide runtime configuration through CLI

• Dynamic update and version control are also needed

– Lack of integration

• There are well-known open source libraries but you should write glue codes

• There are well-known use cases but no simplified API provided

– Lack of reusable and standardized security components

• Vendors use their own log formats

– Everyone writes log parsers again and again

• Response engine requires log merging or compression

– Maybe you don’t want to send hundreds of alarm mails per second

• Poor interoperability

– Data model and API can be standardized per product group

Introduction

Kraken– OSGi based security platform

• Highly configurable

– Interactive Console

– Hot Deploy

– Version and Dependency Control

• Component based Development

– Using iPOJO (injected POJO)

– Managed Lifecycle

– Declarative Services (e.g. Transaction, Web Service)

• Prebuilt Infrastructure and security components

– Infrastructure: Filter, HTTP, JPA

– Web services: JSON, XML, Text, FusionCharts Servlets

– Networking: Syslog, SNMP, JPCAP, DNS

– and more

– Distributed under Apache Software License 2

Introduction

Real world example

Syslog Receiver

Log Handler JPA

Database

Business Logicwith

JsonMethod,XmlMethod

FusionChartsData Service

Jetty

XML ServletJSON Servlet

Web StaticResource

Web Browser

Agents

SNMP4JPerformanceMonitoring

net-snmpd

Kraken Bundles

External Components

Custom Application

Legend

Web based Monitoring System

Introduction

Architecture

JavaSE 1.5+

Apache Felix

iPOJO 1.2.0

Filter

Kraken Core

MavenHTTPInfra

iPOJO

Kernel

Runtime

OSGi R4

Kra

ken S

crip

t API

JPASNMP

JPA

Syslog JSON

ApplicationLayer

Export-Package

Import-Package

Private-Package

Bundle-Version

Bundle-SymbolicName

META-INF

OSGI-INF

OSGi Bundle

iPOJO-Components

Kraken Application Stack

Introduction

Reference– AirSCAN (Wavesoft Inc.)

• Monitor and block wired, wireless, adhoc, wibro, hsdpa, bluetooth devices

• Register wireless devices to AirTight SpectraGuard Enterprise

– WatchCat (NCHOVY Inc.)

• ESM as a Service

• Minimize service downtime

• Highly extensible architecture

• Under development

Kraken APIlearn by example

Kraken API

Class diagram

ScriptContextImpl

<<interface>>

ScriptContext<<interface>>

ScriptInputStream+read(): char+readLine(): string

<<interface>>

Script+setScriptContext()

<<interface>>

ScriptOutputStream+print()+println()

<<interface>>

ScriptFactory+createScript(): Script

ConsoleInputStreamBundleScript

command(String[] args)

ConsoleOutputStream

BundleScriptFactorycalled by reflectionvoid script(String[]args) {}

documented by@ScriptUsage

Kraken API

First script example– Create project using maven

• mvn archetype:generate-DarchetypeGroupId=org.krakenapps-DarchetypeArtifactId=kraken-template-DarchetypeVersion=1.0.0-DgroupId=YOUR_GROUP_ID-DartifactId=YOUR_ARTIFACT_ID-Dversion=YOUR_VERSION-DpackageName=YOUR_DEFAULT_PACKAGE

• Assumptions on this example:

– YOUR_GROUP_ID is org.krakenapps

– YOUR_ARTIFACT_ID is kraken-example

– YOUR_VERSION is 1.0.0

– YOUR_DEFAULT_PACKAGE is org.krakenapps.example

Kraken API

First script example– Maven Configuration

• Edit POM file

– <name>Kraken Example</name>

– <Export-Package>org.krakenapps.example</Export-Package>

– Remove <Private-Package> element

– Remove <Import-Package> element

– Add Kraken API dependency

» <dependency><groupId>org.krakenapps</groupId><artifactId>kraken-api</artifactId><version>1.0.0</version>

</dependency>

– Add <version>1.2.0</version> below the maven-ipojo-plugin element

Kraken API

First script example– iPOJO Configuration

• Edit metadata.xml<ipojo>

<component className="org.krakenapps.example.ExampleScriptFactory"name="exampleScriptFactory" immediate="true" factory="false“>

<provides><property name="alias" type="string" value="example" />

</provides></component><instance component="exampleScriptFactory" />

</ipojo>

• All console command is prefixed with alias

– example.hello means void hello(String[] args) method of a script created from ExampleScriptFactory

• Above configuration declares and instanciates an iPOJO component. See the Apache Felix iPOJO Wiki documentations.

Kraken API

First script example– Edit ExampleScriptFactory.java

package org.krakenapps.example;

import org.krakenapps.api.Script;

import org.krakenapps.api.ScriptFactory;

public class ExampleScriptFactory implements ScriptFactory {

@Override

public Script createScript() {

return new ExampleScript();

}

}

– ScriptFactory constructor can receive BundleContext parameter

• Finds other OSGi services and create Script object with them

• e.g. bundleContext.getServiceReference(interfaceName);

Kraken API

First script example– Edit ExampleScript.java

package org.krakenapps.example;

import org.krakenapps.api.Script;import org.krakenapps.api.ScriptContext;

public class ExampleScript implements Script {private ScriptContext context;

@Overridepublic void setScriptContext(ScriptContext context) {

this.context = context;}

public void hello(String[] args) {context.println("hello kraken");

}}

– Control your components at console• Add command methods as many as you want

Kraken API

First script example– Build

• mvn package

• will generate kraken-example-1.0.0.jar in target directory

– Connect Kraken Console and Install

• telnet localhost 7004

• kraken> bundle.install org.apache.felix org.apache.felix.ipojo 1.2.0download from maven central repository and install

• kraken> bundle.install file:///c:/DIRECTORY/target/kraken-example-1.0.0.jar

• kraken> bundle.listsee id number of bundles and start them

• kraken> bundle.start 1 2

– Test

• kraken> example.hellohello kraken

Kraken Filterruntime filter composition

Kraken Filter

Filter

<<interface>>

Filter+getInputMessageSpecs()+getOutputMessageSpec()+process(Message)+getProperty(String)+setPropety(String, String)+unsetProperty(String)+validateConfiguration()

DefaultFilterimplemented

property operations

ActiveFilter+isRunning()+setRunning()+open()+close()+run()

<<interface>>

FilterChain+process(Message)

<<interface>>

FilterManager+loadFilter()+unloadFilter()+runFilter()+stopFilter()+bindFilter()+unbindFilter()+getFilter()+getInputFilters()+getOutputFilters()+registerFilter()+unregisterFilter()+subscribeFilterEvent()+unsubscribeFilterEvent()+getPropertyKeys()+getProperty()+setProperty()+unsetProperty()

<<interface>>

FilterEventListener+onFilterLoaded()+onFilterUnloading()+onFilterBound()+onFilterUnbinding()+onFilterSet()+onFilterUnset()

Implemented as aniPOJO component

registered as OSGi service

FilterChain implementationis injected by iPOJO

Kraken Filter

Message Specification– Source’s output spec and destination’s input spec have to be matched.

• Bind failed if specification does not match

Message

<<interface>>

MessageSpec+getName()+getDescription()+getLatestVersion()+getVersionRange()

<<interface>>

MessageSpecVersion+getMajorVersion()+getMinorVersion()+isInRange()

<<interface>>

MessageSpecVersionRange+getLowerBound()+getUpperBound()

<<interface>>

Message+getMessageSpec()+headerKeySet()+keySet()+containsHeader()+containsKey()+getHeader()+get()

MessageBulider+setBase(Message)+setHeader(String, Object)+set(String, Object)+build()

Message is immutablefor multithreaded processing

support method chaining

Kraken Filter

Message Flow Example– Provides complex functionality by runtime filter binding

SyslogReceiver SyslogSender

Input Message Specifications

kraken.syslog 1.0kraken.syslog.sender 1.0

Output Message Specification

kraken.syslog 1.0

filter.bind syslogd senderkraken.syslog 1.0 match

filterChain.process(message)

instance.name = senderaddress = 61.250.92.151port = 514encoding = euc-kr

instance.name = syslogdaddress = 192.168.88.88port = 514encoding = utf-8

Syslog Relay with transcoding

Kraken Filter

Filter Script– backed by filter manager

• filter.list

• filter.load [filter class name] [alias]

• filter.unload [alias]

• filter.bind [source alias] [destination alias]

• filter.unbind [source alias] [destination alias]

• filter.status [alias]

– show properties and bind status of the filter

– list all loaded filter instances if alias is omitted

• filter.run [alias] [interval]

– only for active filter

– 1second if interval is omitted

• filter.stop [alias]

– only for active filter

• filter.set [alias] [key] [value]

• filter.unset [alias] [key]

Kraken Filter

Syslog tutorial– Create project using maven

• mvn archetype:generate -DarchetypeGroupId=org.krakenapps-DarchetypeArtifactId=kraken-template-DarchetypeVersion=1.0.0-DgroupId=YOUR_GROUP_ID-DartifactId=YOUR_ARTIFACT_ID-Dversion=YOUR_VERSION-DpackageName=YOUR_DEFAULT_PACKAGE

• Assumptions on this example:

– YOUR_GROUP_ID is org.krakenapps

– YOUR_ARTIFACT_ID is kraken-syslog-example

– YOUR_VERSION is 1.0.0

– YOUR_DEFAULT_PACKAGE is org.krakenapps.example.syslog

Kraken Filter

Syslog tutorial– Maven Configuration

• Edit POM file

– <name>Kraken Syslog Example</name>

– <Export-Package>org.krakenapps.example.syslog</Export-Package>

– Remove <Private-Package> element

– Remove <Import-Package> element

– Add Kraken dependency» <dependency>

<groupId>org.krakenapps</groupId><artifactId>kraken-api</artifactId><version>1.0.0</version>

</dependency><dependency>

<groupId>org.krakenapps</groupId><artifactId>kraken-filter</artifactId><version>1.0.1</version>

</dependency>

Kraken Filter

Syslog tutorial

package org.krakenapps.example.syslog;import org.krakenapps.filter.ActiveFilter;import org.krakenapps.filter.Message;import org.krakenapps.filter.MessageSpec;

public class SyslogReceiver extends ActiveFilter {@Overridepublic void open() throws ConfigurationException {}

@Overridepublic void close() {}

@Overridepublic void run() {}

@Overridepublic MessageSpec getOutputMessageSpec() {

return null;}

}

Kraken Filter

Syslog tutorial– Implement Ouput Message Specification

• "kraken.syslog" message will contain following fields:

– date: received time. java.util.Date type

– message: decoded text string. String type.

– remote_ip: remote ip address, InetAddress type.

– remote_port: remote udp port, Integer type.

– local_ip: local ip address, InetAddress type.

– local_port: local udp port, Integer type

• Decoder will be set by setProperty

@Overridepublic MessageSpec getOutputMessageSpec() {

return new DefaultMessageSpec("kraken.syslog", 1, 0);}

Kraken Filter

Syslog tutorial– Initialize and Finalize

address = InetAddress.getByName((String) getProperty("address"));server_port = Integer.parseInt((String) getProperty("port"));if (server_port == 0)

server_port = DEFAULT_SYSLOG_PORT;

System.out.println("Port: " + server_port);System.out.println("Address: " + address.toString());

socket = new DatagramSocket(server_port, address);socket.setSoTimeout(1000);

buffer = new byte[1024];

socket.close();

close()

open()

Kraken Filter

Syslog tutorial

DatagramPacket packet = new DatagramPacket(buffer, buffer.length);try {

socket.receive(packet);MessageBuilder b = new MessageBuilder(getOutputMessageSpec());b.set("date", new Date());b.set("remote_ip", packet.getAddress());b.set("remote_port", new Integer(packet.getPort()));b.set("local_ip", address);b.set("local_port", new Integer(server_port));b.set("message", new String(packet.getData(), 0, packet.getLength(), charsetName));Message message = b.build();filterChain.process(message);

logger.debug((String) message.get("message"));} catch (SocketTimeoutException e) {} catch (Exception e) {

logger.warn("syslog receive error:", e);}

run()

Kraken Filter

Syslog tutorial– Setup metadata

<ipojo xmlns:filter="org.krakenapps.filter.FilterHandler"><component classname="org.krakenapps.syslog.SyslogReceiver"

immediate="true" architecture="true"><provides /><filter:filter />

</component></ipojo>

Kraken CronTask Scheduling

Kraken Cron

<<interface>>

CronService

+getScheduleList()+getJobList()+registerSchedule(Schedule)+unregisterSchedule(int)

CronConfig

+addEntry(Schedule)+removeEntry()+getEntries()

Scheduler

+start(Map<int, Schedule>)+stop()+insertToQueue(int, Schedule)+deleteFromQueue(int)

Job

+isTimeToDo()+isTimeToDo(Date)+setNextOccurrence()+setNextOccurrence(Date)+run()

NextOccurrenceCalculator

+getNextOccurrence(Schedule, Date)

Schedule

+get(CronField.Type)+getTaskInstanceName()

CronField

+next(int)+first()

CronScript

+list(String[])+queue(String[])+register(String[])+unregister(String[])+run(String[])+runnables(String[])

Kraken Cron

Cron API– How to build Schedule object?

• Run every minute

– Schedule sample = new Schedule.Builder("sample").build();

• Run every day

– Schedule sample = new Schedule.Builder("sample").buildDaily();

• Run every week

– Schedule sample = new Schedule.Builder("sample").buildWeekly();

• Run every month

– Schedule sample = new Schedule.Builder("sample").buildMonthly();

• Run every year

– Schedule sample = new Schedule.Builder("sample").buildYearly();

• 0 0 1 1 3-6 sample

– Schedule sample = new Schedule.Builder("sample").build("0 0 1 1 3-6");

• 0,3,5,9 4-10 */10 * 0 sample

– Schedule sample = new Schedule.Builder("sample")

.set(CronField.Type.DAY_OF_WEEK, "0")

.set(CronField.Type.DAY_OF_MONTH, "*/10")

.set(CronField.Type.HOUR, "4-10")

.set(CronField.Type.MINUTE,"0,3,5,9")

.build();

Kraken Cron

Cron API– How to get CronService reference?

• ServiceReference ref = bundleContext.getServiceReference(CronService.class.getName());CronService cronService = (CronService) bundleContext.getService(ref);

– Schedule Registration

• Schedule schedule = new Schedule.Builder("sample").build();int id = cronService.registerSchedule(schedule);

– Schedule Unregistration

• cronService.unregisterSchedule(id);

Kraken Cron

Cron Script– cron.usage

• print usage

– cron.register [rule] [instance.name]

• cron.register * * * * * sample

• run 'sample' service instance every minutes

– cron.unregister [id]

• unregister schedule

– cron.list

• list all schedules

– cron.queue

• view current state of scheduling queue

– cron.run [instance.name]

• run now

– cron.runnables

• list all active Runnable instances