A user mode implementation of filtering rule managementplane on virtualized networking environment

Network Security Research Institute, National Institute of Information and Communications TechnologyRuo Ando

情報通信研究機構 ネットワークセキュリティ研究所安藤類央

CSEC66 2014-07-04

第 66 回 CSEC ・第 10 回 SPT 合同研究発表会セッション C-5(13:25 - 14:40) http://starbed.nict.go.jp/

Abstract: Towards alternative access control model 仮想ネットワーク環境での新しいアクセス制御モデル

[A] The emergence of network virtualization and related technologies such as SDN and Cloud computing make us face the new challenge of new alternative access control model.

(1) ネットワーク仮想化（抽象化）とその周辺技術（ Software Defined Network, クラウドコンピューティング）の発達により、アクセス制御技術にも革新と修正が求められている。(2) アクセス制御技術の近年の傾向としてはカーネル空間での実装と一元化 (Centrazlization) が問題領域とネットワークの大規模に伴い様々な問題が生じている。

[B] Particularly, besides flexibility, fine-grained traffic engineering functionality for coping with scalability and diversified networks is required for the deployments of SDN and Cloud Computing.

[C] Our architecture leverages NoSQL data store for handling a large scale of filtering rules. By adopting NoSQL, we can achieve scalability, availability and tolerance to network partition. Besides, separating management plane and control plane, we can achieve responsiveness and strong consistency at the same time.

(3) 提案システムには大規模なフィルタリングルールの処理のために NoSQL データストアを用いて、スケーラビリティを確保し、従来の SDN のアーキテクチャを修正し、 management plane を control plane から明示的に分離し、ネットワークのレスポンスを向上させた。

[D] In experiment, we have prototyped a lightweight management plane for IP filtering. Access filtering rules including target IP address, prefix and gateway is represented as radix tree. It is shown that proposed method can achieve reasonable utilization in filtering IP packets

(4) 評価実験では、 radix tree を用いた IP フィルタリングを実装し、 10 ～ 100 以内のフィルタリングルールセットでは許容可能な負荷率で稼動する（ CPU 負荷率では 3% 以内）ことを示した。

Related works: Long-term trend

Centralized access control model

2000 2005 2010 2014

Virtualization Technology (hypervisor)

Cloud computing

Domain Specific / Declarative Language

Open Flow

DSL for IDS and access control

New Security Concerns

DSL for SDN

Rules and conflict management

sHype

Pyretic

Xen

Chimera

ForNox

RuleBricksRemote Attestation

Attack on multi-tenant

アクセス制御の一元化

ネットワーク抽象化

アクセス制御の表現方法の問題

Open vSwitch

vMAC

sandbox xenprobe

subvirt

仮想化、ハイパーバイザー内での AC 一元化

スケーラブル、合成可能な観測システムの構築BGP attack

ネットワークのスライシング仮想化技術を用いた動的解析

ConfAid

BotMiner解析の大規模化

大規模システムでの設定・ Access Ctrl の検査

Flowvisor

HyperDex Ensenble

Avenues of Attack

Sensitive data

Enterprise Network

MissingSecurity Patches

MisconfiguredDatabase

Advanced Attacks

Sensitive Data Leaks

EscalatingUser Privileges

DefaultPasswords

Weak Passwords

Unauthorized Database

WeakPRNG

CDP:Functional & Operational Firewall Pattern - AWS-CloudDesignPatternNemesis: preventing authentication & access control vulnerabilities in web applications, SSYM'09 Proceedings of the 18th conference on USENIX security symposiumDetecting BGP configuration faults with static analysis, NSDI'05 Proceedings of the 2nd conference on Symposium on Networked Systems Design & ImplementationA security enforcement kernel for OpenFlow networks, HotSDN '12 Proceedings of the first workshop on Hot topics in software defined networks

MisconfiguredFiltering

proposed system （設計方針）： building management plane for scalability and manageability

DataStore の適用

Scalability を確保する

一元管理：設定管理

ミスをなくす

Data plane と Control plane は同一ホスト（インスタンス）内で分離しない。

フィルタリング・設定情報の管理用に Management plane を設置して管理。

全ての component を user mode で実装することで deployment のコストを下げ、迅速にする。

Alternative access control model for virtualized computing environment 　仮想ネットワーク• Virtualized Networking environment = SDN + Cloud computing.• Instance is virtualized • Network is also virtualized

hypervisor

VM VM

hypervisor

VM VM

SDN controller Network operating system

Routing Traffic Engineering

Load Balancing

Open Flow Switch Open Flow SwitchData Plane

Control Plane

各ノードは仮想化され、マルチテナントとして同一物理 NIC上に配置される。

ネットワークも仮想化され、フィルタリングとマッチング処理が分離一元化される。

Open vSwitch. B. Pfaff, J. Pettit, K. Amidon, M. Casado, T. Koponen, an d S. Shenker. Extending Networking into the Virtualization Layer. In HotNets, 2009

Operational / Functional FW

Amazon EC2 Design Pattern

What is SDN and network virtualization ?Myth: “SDN is network virtualization”

x86 / ARM

Virtualization Layer

Windows Linux

Open Flow

Virtualization Or Slicing

NOX NOX

CPU, Hardisk, PIC, IO

X86 instruction set

Xen, QEMU, etc

Windows Linux

Hardware Resources

Abstraction layer

Virtualization Layer

slice slice

Bandwidth, CPU, FIB

OpenFlow

FlowVisor

Controller Contoller

Definition of a slice• Slice is a set of flows (called flowspace) running on a topology of switches.https://www.clear.rice.edu/comp529/.../tutorial_4.pdf

設計上の問題：仮想化環境とスケーラビリティ

Centralized access control model

2000 2005 2010 2014

Virtualization Technology (hypervisor)

Cloud computing

Domain Specific / Declarative Language

Open Flow

DSL for IDS and access control

New Security Concerns

DSL for SDN

Rules and conflict management

sHype

Pyretic

Xen

Chimera

ForNox

RuleBricksRemote Attestation

Attack on multi-tenant

アクセス制御の一元化

ネットワーク抽象化

アクセス制御の表現方法の問題

Open vSwitch

vMAC

sandbox xenprobe

subvirt

仮想化、ハイパーバイザー内での AC 一元化

スケーラブル、合成可能な観測システムの構築BGP attack

ネットワークのスライシング仮想化技術を用いた動的解析

ConfAid

BotMiner解析の大規模化

大規模システムでの設定・ Access Ctrl の検査

Flowvisor

HyperDex Ensenble

“when virtual is harder than real”drawbacks of virtualized networkTal Garfinkel , Mendel Rosenblum, When virtual is harder than real: Security challenges in virtual machine based computing environments, HotOS 2005

Scalability. Growth in physical machines is ultimately limited by setup time and bounded by organization‘s capital equipment budget. In contrast creating a new VM is as easy as copying file. Users will frequently have several or even dozens of special purpose VMs . Thus ， total number of VMs in an organization can grow at an explosive rate ． Rarely all administrative tasks completely automated.

Diversity. Many IT organizations tackle security problems by enforcing homogenity. all machines must run the most current patched software. This creates a range of problems as one must try and maintain patches or other protection for a wide range of OS and deal with the risk posed by having many unpatched machines on the network.

Access Control should be centralized !

CloudPolice: Taking access control out of the network Lucian Popa, Minlan Yu, Steven Y. Ko, Ion Stoica, Sylvia Ratnasamy 9th ACM Workshop on Hot Topics in Networks (HotNets-IX). Monterey, CA, October 2010.Jonathan M McCune, Stefan Berger, Trent Jaeger, Reiner Sailer: Shamon -- A System for Distributed Mandatory Access Control. 22nd Annual Computer Security Applications Conference (ACSAC), Miami Beach, Florida, December 2006

Design requirement: fine grained traffic functioning for scalability, diversity and flexibility.[1] Scalability and diversity: Garfinkel pointed that creating a new 　 virtual instance is far easier than physical environment. the rapid and unpredictable growth can exacerbate management tasks and in worse case the impact of catastorophic events can be multiplied where all instances should be patched. Enforcing homogenity is difficult in the situation that users can have their own special purpose VM easily without expensive cost, like copying files.

[2] Flexibility: In SDN, networks are diversified, programmable and elastic. For a long period, from active networks to advanced network technologies like cloud and SDN, one of the general goals of net working research has been arrived at a network which is flexible.

[3] Fine-grained traffic functioning: commercial corporations ， private Enterprises and universities emplos datacenters to run variety of applications and cloud based services. Their study reveals that existing traffic engineering perform 15%to 20% worse than the optimal solution.

MicroTE: fine grained traffic engineering for data centers, CoNEXT '11 Proceedings of the Seventh COnference on emerging Networking EXperiments and Technologies

Lucian Popa, Ion Stoica, Sylvia Ratnasamy: Rule-based Forwarding(RBF): Improving Internet’s flexibility and security. HotNets 2009

Tradeoffs between manageability and performance

"Logically centralized?: state distribution trade-offs in software defined networks", Dan Levin, Andreas Wundsam, Brandon Heller, Nikhil Handigol and Anja Feldmann, HotSDN '12 Proceedings of the first workshop on Hot topics in software defined networks

Controller component choices:[1] Strongly consistent – controller components always operate on the same world view. Imposes delay and overhead.[2] Eventually consistent – controller components incorporate information as it becomes available but may make decisions on different world views.http://www.richardclegg.org/node/21

C A

PNoSQLRDBMS

Consistency Availability

Tolerance to networkpartition

CAP Theorem (Eric Brewer 2000)

Enforced Consistency Eventual Consistency パケットフィルタなどの処理では

Strongly Consistent であることが求められる（望ましい）

NoSQL を用いることでA (availability)P (Tolerance to network partition) S (Scalability)が達成される。

Basic SDN architecture and proposed system

Node (VM)

Node (VM)

Node (VM)

Flow Table

ControllerSecure Channel

Node (VM)

Node (VM)

Node (VM)

Filtering rule Table

Data store

match

match

判定処理 (match) はノード側で行い、パケットは転送しない。→　これにより、フィルタリングの一元管理を達成しつつ、 Control/Management Plane の負荷を下げる。

Ingress packets

Ingress packets

Data plane Control plane

Control and Data plane Management plane

VCRIB: Virtualized rule management in the cloud Masoud Moshref, 　 Minlan Yu, Abhishek Sharma, Ramesh Govindan the 4th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud). Boston, 　MA, June 2012.

Basic SDN

提案手法

Adopting datastore (NoSQL) on management plane

auto_ptr<mongo::DBClientCursor> cursor = client.query(ns, mongo::BSONObj());

while(cursor->more()) { mongo::BSONObj p = cursor->next(); mongo::OID oid = p["_id"].OID(); string dest = p["dest"].str(); int mask = p["mask"].numberInt(); string gateway = p["gateway"].str(); const char *p0 = dest.c_str(); const char *p1 = gateway.c_str(); add_rtentry(p0, mask, p1); int res; res = find_route(dstAddress); if(res==0) printf("route find \n"); /* flush entry /*

rm_rtentry(p0, mask);

{"_id": "$oid":"53370eaeb1f58908a9837910"

"dest":"10.0.0.0","mask": 8,"gateway":"192.168.0.2"}

Radix tree のエントリを動的に読み出すことでリモート (management plane) から任意のタイミングでフィルタリングルールを変更できるようにする。

Filtering rule は BSON (JSON) で記述

a radix tree (also patricia trie or radix trie or compact prefix tree) is a space-optimized trie data structure where each node with only one child is merged with its parent.

14 entry.addr = ntohl(addr dst.s addr);15 entry.prefix len = 32;17 radix tree<rtentry, in addr>::iterator it;1819 it = rttable.longest match(entry);20 if (it == rttable.end()) f21 std::cout << ‘‘no route to ‘‘ << dst << std::endl;22 return 1;

Experimental result on Amazon VPC

We compiled our system on ubuntu12 LTS with Linux kernel 3.2.0. proposed system is hosted on Intel Xeon E5645 with 2.4 GHZ clock.

ルール数が１０～１００以内であれば、 CPU 負荷率（３％以内）、コンテキストスイッチ（５０００回以内）とも許容範囲内で稼動。

設定した環境内では、 1000 ～ 10000 のルールを用いるケースは稀なため、提案手法は妥当。

vNIC1 vNIC2

Bridge

IP capture

1

2

3

MongoDB

4

5

8

7

8

Radix Module6

0

Management plane Control plane

Python module

パケットキャプチャはユーザモードでのpcap を利用。ルールはPython を用いて設定

Further works

Centralized access control model

2000 2005 2010 2014

Virtualization Technology (hypervisor)

Cloud computing

Domain Specific / Declarative Language

Open Flow

DSL for IDS and access control

New Security Concerns

DSL for SDN

Rules and conflict management

sHype

Pyretic

Xen

Chimera

ForNox

RuleBricksRemote Attestation

Attack on multi-tenant

アクセス制御の一元化

ネットワーク抽象化

アクセス制御の表現方法の問題

Open vSwitch

vMAC

sandbox xenprobe

subvirt

仮想化、ハイパーバイザー内での AC 一元化

スケーラブル、合成可能な観測システムの構築BGP attack

ネットワークのスライシング仮想化技術を用いた動的解析

ConfAid

BotMiner解析の大規模化

大規模システムでの設定・ Access Ctrl の検査

Flowvisor

HyperDex Ensenble

Conclusions

[A] The emergence of network virtualization and related technologies such as SDN and Cloud computing make us face the new challenge of new alternative access control model.

(1) ネットワーク仮想化（抽象化）とその周辺技術（Software Defined Network, クラウドコンピューティング）の発達により、アクセス制御技術にも革新と修正が求められている。(2) アクセス制御技術の近年の傾向としてはカーネル空間での実装と一元化 (Centrazlization) が問題領域とネットワークの大規模に伴い様々な問題が生じている。

[B] Particularly, besides flexibility, fine-grained traffic engineering functionality for coping with scalability and diversified networks is required for the deployments of SDN and Cloud Computing.

[C] Our architecture leverages NoSQL data store for handling a large scale of filtering rules. By adopting NoSQL, we can achieve scalability, availability and tolerance to network partition. Besides, separating management plane and control plane, we can achieve responsiveness and strong consistency at the same time.

(3) 提案システムには大規模なフィルタリングルールの処理のためにNoSQLデータストアを用いて、スケーラビリティを確保し、従来のSDNのアーキテクチャを修正し、management planeをcontrol planeから明示的に分離し、ネットワークのレスポンスを向上させた。

[D] In experiment, we have prototyped a lightweight management plane for IP filtering. Access filtering rules including target IP address, prefix and gateway is represented as radix tree. It is shown that proposed method can achieve reasonable utilization in filtering IP packets

(4) 評価実験では、radix treeを用いた IPフィルタリングを実装し、10～100以内のフィルタリングルールセットでは許容可能な負荷率で稼動する（CPU負荷率では3%以内）ことを示した。

www.slideshare.net/RuoAndohttps://github.com/RuoAndo/cBridge