what's new and what's next in rudder

Whats new and whats next in Rudder

Nicolas CHARLES Co-founder and COO

@nico_charles

2

Agenda

Rudder news since last camp

Whats new in Rudder 4.1, 4.2 and 4.3

Future direction

3

Rudder versions

Currently supported versions 4.1.x current ESR* version

4.2.x current latest version Will be supported for 3 months after next version is released

(so until May 2018)

4.3.x next version Scheduled for release in February 2018

* ESR = Extended Support Release

2015 2016 2017 2018Jun Jan Oct Mar Sep Feb

3.1 3.2 4.0 4.1 4.2 4.3

4

Microsoft Powershell DSC Agent

4.2Manage Windows Systems using Powershell DSC

Native agent for Windows Uses Microsoft Powershell DSC

Requires Powershell 4 or more

5

Microsoft Powershell DSC Agent

4.2Generic methods for DSC, Classic or both Agents

6

New generation method: Directive by Directive

Directives are not merged together anymore! New Policy Generation type: SEPARATED Mix audit and enforce mode for same Technique on a Node Several versions at the same time on a node

4.3

7


Directives are not merged together anymore! Most techniques have been rewritten to support this feature

New version (but you can upgrade them one directive at a time)

separated

Generate one folder per directive Path: TechniqueName/TechniqueVersion_DirectiveID

4.3

8


Technically, RudderUniqueID placeholder In Technique bundle names/function names In resulting class to avoid collisions

Hooks: One time action before and after Directives For global actions like getting the repositories PGP keys only once

4.3

9

Techniques Parameters

Defines parameters within the Technique Editor Better re-usability

4.3

10

Techniques Parameters

Defines parameters within the Technique Editor Define parameter during Directive creation

4.3

11

Node properties

Node properties can now be JSON values:datacenter = { "id": "FRA1", "name": "Colo 1, Paris", "location": "Paris, France", "dns_suffix": "paris.example.com"}

Access properties in any directive field:${node.properties[datacenter]}

${node.properties[datacenter][id]}

Use default values:${node.properties[datacenter][id] | default = "UK2" }

${node.properties[netbios_name] | default = ${rudder.node.hostname} }

${node.properties[datacenter][name] | default = """value with "quotes" if necessary""" }

4.1

12

Node properties

Import automatically properties on nodes from third-party REST application

Datasource plugin Drive behaviour from external source Specific type of Node Property

4.1

13

Node properties


4.1

14

Node properties


Can be global or on a node by node basis Add data in headers

4.1

15

Node properties


Extract from received JSON relevant information

4.1

16

Node properties


Customize update frequency

4.1

17

Node properties


Define what happens when the API doesnt answer

4.1

18

Node properties

Agent searches for optional properties files

/var/rudder/local/properties.d/*.json

Add new properties or override existing properties defined on Rudder

Example:

Results in :"sysctls_postgresql":{"kernel.shmmax":"5368709120"}

4.1

On the node side

"sysctls_postgresql": { "kernel.shmmax":"5368709120"}

On the server side

"sysctls_postgresql": { "kernel.shmall":"903330", "kernel.shmmax":"3700041320"}

Override node properties locally

19

JSON everywhere

ncf methods

variable_dictvariable_dict_

from_fle

Import JSON at runtime 4.1

20

JSON everywhere

4.1Merge JSON at runtime

21

Tags everywhere!

4.1Tags on Directives and Rules to classify and filter

22

A new API on relay servers

Central server

Node Node Node

TCP communication (port 5309)File metadataFile contents

Authentifcation + encryption (TLS)

TCP/UDP communication (ports 443 and 514)

Protocols: HTTPS, syslog

Node Node Node

Isolated network zone

Relay server

Inventory+ Reports

Confgurationpolicy

23

... RELAY API


Relay server

Node Node

UI REST API

... RELAY API

Central server

RELAY APIRELAY API

Trigger agent runScenario 1:Trigger agent runs remotely, including

via firewalls.4.1

24

... RELAY API


Relay server

Node 1 Node 2

UI REST API

... RELAY API

Central server

RELAY APIRELAY API

File sharedwith metadata

Scenario 2:Share files from one

node to another.

In the same networkor not (via relays).

... RELAY API

Relay server

sharedfle_to_node(node 2, db.sql, /var/share/db.sql, 3 days)

sharedfle_from_node(node 1, db.sql, /var/share/db.sql)ncf methods

4.1

25

Hooks

4.1Customize behaviour on the server

26

Hooks

4.3Extend inventory agent side Inventory runs hooks in /var/rudder/hooks.d or C:\Program

Files\Rudder\hooks.d Executable scripts, owned by current user or root, and not world writable Script must return valid JSON Added in inventory tag CUSTOM_PROPERTIES

Available in Node Properties on the Rudder Server Can be used to create Groups Available in API

27

Improved performance

Improved UI performance New graph rendering library All Web resources are cached Compress all data from Web Interface

Better Agent performance 40% faster in normal usage, up to 20 times faster with large policies

Slightly faster policy generation

4.1

28

Agent

Lighter agent Perl is no longer packaged within the Agent

SystemD support

Timing on the CLI output

Dropped the old cfengine network protocol

4.3

29

Miscellaneous

Groups of groups

Node lifecycle

Renaming of ncf generic method And a tool to automatically update the generic method call

Same versioning for Rudder & ncf

4.3

30

Plugins

Branding: Customize Rudder UI Backgrounds and font colours Login page Logos Title text

31

Plugins

Precise ACLs on API Rights per token on any REST API endpoint Token expiration date Maps user permissions to tokens

What can we define? AclPath : segments, separated by /

Segment is either a String (api, nodes, rules, etc) Wildcard * , anywhere as a segment Double wildcard ** , only at the end, matches anything

HttpAction (GET, POST, PUT, DELETE) Anything that is not authorized is denied

4.3

32

Plugins

Examples ALLOW api/nodes/** , GET

Permits to read all in the nodes API But no changes at all

ALLOW api/nodes, GET Permits to list nodes (including searches), but not the pending nodes

ALLOW api/directives/7dd68892-6820-4f85-8e44-a7cc820dd06e , POST Edit only directive with id 7dd68892-6820-4f85-8e44-a7cc820dd06e

ALLOW api/directives/*/check, POST Only permits to valid that a change is valid

4.3

33

Plugins

Centreon: Automatically configure monitoring on systems

Node

UI REST API

ncf RELAY API

Central server

RELAY APIRELAY API

1 - Synchronize all nodes in Centreon

Plugin

2 - Configure node

3 - Configure hosttemplate

34

Plugins

iTop: CMDB integration Export inventories to iTop Import properties from iTop

Drive policies from CMDB and external data Export Directives and Compliance

Measure impact of non-compliance

35

Plugins

Reporting Generate compliance reports over a period of time

In development progress

36

Plugins

Reporting Defines Rules/Groups/Nodes and a period Select what to display

37

Bug classification

38

Bug classification 3 parameters

39


User visibility: use case impacted by issue First impression even before Rudder installation Getting started during demo, first install or basic usage of simple Techniques Operational usage of Technique Editor, advanced Techniques, Rudder settings Infrequent complex configurations, third-party integration

40


41


Severity: Critical Prevent main usage of Rudder, can cause data loss no workaround Major Prevent usage of a part of Rudder no easy workaround Minor Something is misleading or with an easy workaround Trivial No functional impact, but it would be nicer if it were fixed.

42


43


Effort required: Small This issue can be solved in less than a day Medium It can be fixed in a reasonable amount of time Large This issue is complex, needs some thoughts and time (about a week) Very large This issue is so complex that we cannot estimate its duration

(several weeks to months)

44

Bug classification - Priority

These information are reviewed, and a priority is computed from these

From 0 (lower priority) to about 150 (the top priority) Weighted based on user visibility and severity Biased toward smallest effort and oldest bugs

45

http://faq.rudder-project.org

New FAQ

46

Agenda

The future

47

Client Server communication

Two steps policy update Validation on the client side Ensure complete consistency Much faster policy generation

48

Client Server communication

Drop syslog protocol Send reports via HTTPS Minimize impact of agent on nodes Improve performances and network usage

49

Web Interface

Customize Dashboard

Customize columns in tables

Improve Group page Improve search interface and group creation

Manage Users within the UI

50

Future plugins (planned + ideas)

Sync databetween

Rudder servers

Ideas of plugins

Advanced accesscontrol

(OrBAC)

High availabilityfor Rudder server

Ramp uppolicies forprogressive

rollouts

51

Rudder Ambassador Program

Rudder Ambassador Program for exceptional Rudder contributors To be announced

52

Rudder.io

New Domain name Rudder.io Focus on the Rudder brand

Whats new and whats next in Rudder

Questions ?

Nicolas CHARLES Co-founder and COO

@nico_charles

Slide 1Slide 2Slide 3Slide 4Slide 5Slide 6Slide 7Slide 8Slide 9Slide 10Slide 11Slide 12Slide 13Slide 14Slide 15Slide 16Slide 17Slide 18Slide 19Slide 20Slide 21Slide 22Slide 23Slide 24Slide 25Slide 26Slide 27Slide 28Slide 29Slide 30Slide 31Slide 32Slide 33Slide 34Slide 35Slide 36Slide 37Slide 38Slide 39Slide 40Slide 41Slide 42Slide 43Slide 44Slide 45Slide 46Slide 47Slide 48Slide 49Slide 50Slide 51Slide 52Slide 53

what's new and what's next in rudder

Technology