automating network and service conﬁguration using netconf and...

Automating Network and Service ConfigurationUsing NETCONF and YANG

Stefan WallinLulea University of Technology

[email protected]

Claes WikstromTail-f Systems [email protected]

Abstract

Network providers are challenged by new requirementsfor fast and error-free service turn-up. Existing ap-proaches to configuration management such as CLIscripting, device-specific adapters, and entrenched com-mercial tools are an impediment to meeting these new re-quirements. Up until recently, there has been no standardway of configuring network devices other then SNMPand SNMP is not optimal for configuration management.The IETF has released NETCONF and YANG which arestandards focusing on Configuration management. Wehave validated that NETCONF and YANG greatly sim-plify the configuration management of devices and ser-vices and still provide good performance. Our perfor-mance tests are run in a cloud managing 2000 devices.

Our work can help existing vendors and serviceproviders to validate a standardized way to build con-figuration management solutions.

1 Introduction

The industry is rapidly moving towards a service-oriented approach to network management where com-plex services are supported by many different systems.Service operators are starting a transition from managingpieces of equipment towards a situation where an opera-tor is actively managing the various aspects of services.Configuration of the services and the affected equip-ment is among the largest cost-drivers in provider net-works [9]. Delivering valued-added services, like MPLSVPNS, Metro Ethernet, and IP TV is critical to the prof-itability and growth of service providers. Time-to-marketrequirements are critical for new services; any delay inconfiguring the corresponding tools directly affects de-ployment and can have a big impact on revenue. In re-cent years, there has been an increasing interest in find-ing tools that address the complex problem of deployingservice configurations. These tools need to replace the

current configuration management practices that are de-pendent on pervasive manual work or ad hoc scripting.Why do we still apply these sorts of blocking techniquesto the configuration management problem? As Enck [9]points out, two of the primary reasons are the variationsof services and the constant change of devices. Theseunderlying characteristics block the introduction of au-tomated solutions, since it will take too much time toupdate the solution to cope with daily changes. We willillustrate that a NETCONF [10] and YANG [4] based so-lution can overcome these underlying challenges.

Service providers need to be able to dynamically adoptthe service configuration solutions according to changesin their service portfolio without defining low level de-vice configuration commands. At the same time, we needto find a way to remove the time and cost involved in theplumbing of device interfaces and data models by au-tomating device integration. We have built and evaluateda management solution based on the IETF NETCONFand YANG standards to address these configuration man-agement challenges. NETCONF is a configuration man-agement protocol with support for transactions and dedi-cated configuration management operations. YANG is adata modeling language used to model configuration andstate data manipulated by NETCONF. NETCONF waspioneered by Juniper which has a good implementationin their devices. See the work by Tran [23] et. al forinteroperability tests of NETCONF.

Our solution is characterized by the following keycharacteristics:

1. Unified YANG modeling for both services and de-vices.

2. One database that combines device configurationand service configuration.

3. Rendering of northbound and southbound interfacesand database schemas from the service and devicemodel. Northbound are the APIs published to users

of NCS, be it human or programmatic interfaces.Southbound is the integration point of managed de-vices, for example NETCONF.

4. A transaction engine that handles transactions fromthe service order to the actual device configurationdeployment.

5. An in-memory high-performance database.

To keep the service and device model synchronized,(item 1 and 2 above), it is crucial to understand how aspecific service instance is actually configured on eachnetwork device. A common problem is that when youtear down a service you do not know how to clean up theconfiguration data on a device. It is also a well-knownproblem that whenever you introduce a new feature ora new network device, a large amount of glue code isneeded. We have addressed this again with annotatedYANG models rather then adaptor development. So forexample, the YANG service model renders a northboundCLI to create services. From a device model in YANGwe are actually able to render the required Cisco CLIcommands and interpret the response without the needfor the traditional Perl and Expect scripting. Currentlyour solution can integrate without any plumbing.

It is important to address the configuration manage-ment problem using a transactional approach. The trans-action should cover the whole chain including the indi-vidual devices. Finally, in order to manipulate config-uration data for a large network and many service in-stances we need fast response to read and write oper-ations. Traditional SQL and file-based database tech-nologies fall short in this category. We have used an in-memory database journaled to disk in order to addressperformance and persistence at the same time.

The objectives of this research are to determinewhether these new standards can help to eliminate the de-vice integration problem and provide a service configu-ration solution utilizing automatically integrated devices.We have studied challenges around data-model discov-ery, interface versioning, synchronization of configura-tion data, multi-node configuration deployment, trans-actional models, and service modeling issues. In orderto validate the approach we have used simulated scenar-ios for configuring load balancers, web servers, and websites services. Throughout the use-cases we also illus-trate the possibilities for automated rendering of Com-mand Line interfaces as well as User Interfaces fromYANG models.

Our studies show that a NETCONF/YANG based con-figuration management approach removes unnecessarymanual device integration steps and provides a platformfor multi-device service configurations. We see thatproblems around finding correct modules, loading them

and creating a management solution can largely be auto-mated. In addition to this, the transaction engine in oursolution combined with inherent NETCONF transactioncapabilities resolves problems around multi-device con-figuration deployment.

We have run performance tests with 2000 devices in anAmazon cloud to validate the performance of NETCONFand our solution. Based on these tests we see that thesolution scales and NETCONF provides a configurationmanagement protocol with good performance.

2 Introduction to NETCONF and YANG

The work with NETCONF and YANG started as a resultof an IAB workshop held in 2002. This is documentedin RFC 3535 [18].

“The goal of the workshop was to continue theimportant dialog started between network op-erators and protocol developers, and to guidethe IETFs focus on future work regarding net-work management.”

The workshop concluded that SNMP is not beingused for configuration management. Operators putforth a number of requirements that are important fora standards-based configuration management solution.Some of the requirements were:

1. Distinction between configuration data and data thatdescribes operational state and statistics.

2. The capability for operators to configure the net-work as a whole rather than individual devices.

3. It must be easy to do consistency checks of config-urations.

4. The availability of text processing tools such as diff,and version management tools such as RCS or CVS.

5. The ability to distinguish between the distributionof configurations and the activation of a certain con-figuration.

NETCONF addresses the requirements above. The de-sign of NETCONF has been influenced by proprietaryprotocols such as Juniper Networks JUNOScript appli-cation programming interface [14].

For a more complete introduction see the Communi-cations Magazine article [19] written by Schonwalder etal.

2

2.1 NETCONF

The Network Configuration Protocol, NETCONF, is anIETF network management protocol and is published inRFC 4741. NETCONF is being adopted by major net-work equipment providers and has gained strong industrysupport. Equipment vendors are starting to support NET-CONF on their devices, see the NETCONF presentationby Moberg [16] for a list of public known implementa-tions.

NETCONF provides mechanisms to install, manipu-late, and delete the configuration of network devices. Itsoperations are realized on top of a simple Remote Pro-cedure Call (RPC) layer. The NETCONF protocol usesXML based data encoding for the configuration data aswell as the protocol messages. NETCONF is designedto be a replacement for CLI-based programmatic inter-faces, such as Perl + Expect over Secure Shell (SSH).NETCONF is usually transported over the SSH protocol,using the “NETCONF” sub-system and in many waysit mimics the native proprietary CLI over SSH inter-face available in the device. However, it uses structuredschema-driven data and provides detailed structured er-ror return information, which the CLI cannot provide.

NETCONF has the concept of logical data-stores suchas “writable-running” or “candidate” (Figure 1). Opera-tors need a way to distribute changes to the devices andvalidate them locally before activating them. This is in-dicated by the two bottom options in Figure 1 where con-figuration data can be sent to candidate databases in thedevices before they are committed to running in produc-tion applications.

All NETCONF devices must allow the configurationdata to be locked, edited, saved, and unlocked. In ad-dition, all modifications to the configuration data mustbe saved in non-volatile storage. An example from RFC4741 that adds an interface named “Ethernet0/0” to therunning configuration, replacing any previous interfacewith that name is shown in Figure 2.

2.2 YANG

YANG is a data modeling language used to model con-figuration and state data. The YANG modeling lan-guage is a standard defined by the IETF in the NETMODworking group. YANG can be said to be tree-structuredrather than object-oriented. Configuration data is struc-tured into a tree and the data can be of complex typessuch as lists and unions. The definitions are containedin modules and one module can augment the tree in an-other module. Strong revision rules are defined for mod-ules. Figure 3 shows a simple YANG example. YANGis mapped to a NETCONF XML representation on thewire.

runningautomatic-save

edit-configcopy-config

WRITABLE-RUNNING

running

copy-config


WRITABLE-RUNNING + STARTUP

startup

candidate

commit


CANDIDATE

running

automatic-save

candidate

commit


CANDIDATE + STARTUP

running

copy-config

startup

Figure 1: NETCONF Datastores

<rpc message-id="101"

xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">

<edit-config>

<target>

<running/>

</target>

<config

xmlns:xc="urn:ietf:params:xml:ns:netconf:base:1.0">

<top xmlns="http://example.com/schema/1.2/config">

<interface xc:operation="replace">

<name>Ethernet0/0</name>

<mtu>1500</mtu>

<address>

<name>192.0.2.4</name>

<prefix-length>24</prefix-length>

</address>

</interface>

</top>

</config>

</edit-config>

</rpc>

<rpc-reply message-id="101"

xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">

<ok/>

</rpc-reply>

Figure 2: NETCONF edit-config Operation

3

module acme-system {namespace"http://acme.example.com/system";prefix "acme";organization "ACME Inc.";contact "[email protected]";description"The ACME system.";

revision 2007-11-05 {description "Initial revision.";

}container system {leaf host-name {

type string;}leaf-list domain-search {type string;

}list interface {key "name";leaf name {

type string;}leaf type {

type enumeration {enum ethernet;enum atm;

}}

leaf mtu {type int32;

}must ‘‘ifType != ’ethernet’ or ‘‘+‘‘(ifType = ’ethernet’ and ‘‘ +‘‘mtu = 1500)‘‘ {}

...

Figure 3: YANG Sample

YANG also differs from previous network manage-ment data model languages through its strong supportof constraints and data validation rules. The suitabilityof YANG for data models can be further studied in thework by Xu et. al [24].

3 Our Config Manager Solution - NCS

3.1 OverviewWe have built a layered configuration solution, NCS,Network Configuration Server. See Figure 4. The De-vice Manager manages the NETCONF devices in the

ConfigDatabase

RenderingEngine

ServiceManager

DeviceManager

NETCONF

ServiceLogic

CLI, Web UI

YANGService ModelsDevice Models

Figure 4: NCS - The Configuration Manager

network and heavily leverage the features of NETCONFand YANG to render a Configuration Manager from theYANG models. At this layer, the YANG models repre-sent the capabilities of the devices and NCS provides thedevice configuration management capabilities.

The Service Manager in turn lets developers addYANG service models. For example, it is easy to repre-sent end-to-end connections over L2/L3 devices or websites utilizing load balancers and web servers. The mostimportant feature of the Service Manager is to transforma service creation request into the corresponding deviceconfigurations. This mapping is expressed by definingservice logic in Java which basically does a model trans-formation from the service model to the device models.

The Configuration Database, (CDB), is an in-memorydatabase journaled to disk. CDB is a special-purposedatabase that targets network management and the in-memory capability enables fast configuration valida-tion and performs diffs between running and candidatedatabases. Furthermore the database schema is directlyrendered from the YANG models which removes theneed for mapping between the models and for exam-ple a SQL database. A fundamental problem in net-work management is dealing with different versions ofdevice interfaces. NCS is able to detect the device in-terfaces through its NETCONF capabilities and this in-formation is used by CDB to tag the database with re-vision information. Whenever a new model revision isdetected, NCS can perform a schema upgrade operation.CDB stores the configuration of services and devices andthe relationships between them. NETCONF defines ded-icated operations to read the configuration from devicesand this drastically reduces the synchronization and rec-onciliation problem.

Tightly connected to CDB is the transaction manager

4

web site

IPPortURL

Web Serverwww1

Listeners: {IP, Port}

Web Serverwww2


Web Serverwww3



Load balancerlb

Backend. IP.Port

Profile

Figure 5: The Example

which manages every configuration change as a transac-tion. Transactions include all aspects from the servicemodel to all related device changes.

At this point it is important to understand that theNETCONF and NCS approach to configuration manage-ment does not use a push and pull approach to versionedconfiguration files. Rather, it is a fine-grained transac-tional view based on data models.

The Rendering Engine renders the database schemas,a CLI, and a Web UI from the YANG models. In this waythe Device Manager features will be available withoutany coding.

3.2 The ExampleThroughout the rest of this paper we will use an exam-ple that targets configuration of web-sites across a loadbalancer and web servers. See Figure 5.

The service model covers the aspects of a web site; IPAddress, Port, and URL. Whenever you provision a website you refer to a profile which controls the selection ofload balancers and web servers. A web site allocates alistener on the load balancer which in turn creates back-ends that refer to physical web servers. So when provi-sioning a new web site you do not have to deal with theactual load balancer and web server configuration. Youjust refer to the profile and the service logic will config-ure the devices. The involved YANG models are :

• website.yang : the service model for a web site, itdefines web site attributes like url, IP Address, port,and pointer to profile.

• lb.yang : the device model for load balancers, itdefines listeners and backends where the listeners

refers to the web site and backends to the corre-sponding web servers.

• webserver.yang : the device model for a physicalweb server, it defines listeners, document roots etc.

The devices in our example are:

– Load Balancer : lb

– Web Servers : www1, www2, www3

3.3 The Device ManagerThe Device Manager layer is responsible for config-uring devices using their specific data-models and in-terfaces. The NETCONF standard defines a capabil-ity exchange mechanism. This implies that a devicereports its supported data-models and their revisionswhen a connection is established. The capability ex-change mechanism also reports if the device supports a<writable-running> or <candidate> database.

After connection the Device Manager can then use theget-schema RPC, as defined in the netconf-monitoringRFC [20] to get the actual YANG models from all thedevices. NCS now renders northbound interfaces such asa common CLI and Web UI from the models. The NCSdatabase schema is also rendered from the data-models.

The NCS CLI in Figure 6 shows the discov-ered capabilities for device “www1”. We see thatwww1 supports 6 YANG data-models, interfaces,webserver, notif, and 3 standard IETF modules. Fur-thermore the web-server supports NETCONF featureslike confirmed-commit, rollback-on-error andvalidation of configuration data.

In Figure 7 we show a sequence of NCS CLI com-mands that first uploads the configuration from all de-vices and then displays the configuration from the NCSconfiguration database. So with this scenario we showthat we could render the database schema from theYANG models and persist the configuration in the con-figuration manager.

Now, let’s do some transaction-based configurationchanges. The CLI sequence in Figure 8 starts a trans-action that will update the ntp server on www1 and theload-balancer. Note that NCS has the concept of a can-didate database and a running. The first represents thedesired configuration change and the running databaserepresents the actual configuration of the devices. At theend of the sequence in Figure 8 we use the CLI command‘‘compare running brief’’ to show the differencebetween the running and the candidate database. This iswhat will be committed to the devices. Note that we doa diff and only send the diff. Our in-memory databaseenables good performance even for large configurationsand large networks.

5

ncs> show ncs managed-device www1 capability <RET>

URI REVISION MODULE

------------------------------------------------------------------------------------------------------

candidate:1.0 - -

confirmed-commit:1.0 - -

confirmed-commit:1.1 - -

http://acme.com/if 2009-12-06 interfaces

http://acme.com/ws 2009-12-06 webserver

http://router.com/notif - notif

rollback-on-error:1.0 - -

urn:ietf:params:netconf:capability:notification:1.0 - -

urn:ietf:params:xml:ns:yang:ietf-inet-types 2010-09-24 ietf-inet-types

urn:ietf:params:xml:ns:yang:ietf-yang-types 2010-09-24 ietf-yang-types

urn:ietf:params:xml:ns:yang:ietf-netconf-monitoring 2010-06-22 ietf-netconf-monitoring

validate:1.0 - -

validate:1.1 - -

writable-running:1.0 - -

xpath:1.0 - -

Figure 6: NETCONF Capability Discovery

ncs> request ncs sync direction from-device <RET>

...

ncs> show configuration ncs \

managed-device www1 config <RET>

host-settings {

syslog {

server 18.4.5.6 {

enabled;

selector 1;

}

...

}

ncs> show configuration ncs \

managed-device lb config <RET>

lbConfig {

system {

ntp-server 18.4.5.6;

resolver {

search acme.com;

nameserver 18.4.5.6;

}

}

}

Figure 7: Synchronize Configuration Data from Devices

In the configuration scenarios shown in Figure 8 weused the auto-rendered CLI based on the native YANGmodules that we discovered from the devices. So it givesthe administrator one CLI with transactions across thedevices, but still with different commands for different

ncs% set ncs managed-device \

www1 config host-settings ntp server 18.4.5.7 <RET>


lb config lbConfig system ntp-server 18.4.5.7 <RET>

ncs% compare running brief <RET>

ncs {

managed-device lb {

config {

lbConfig {

system {

- ntp-server 18.4.5.6;

+ ntp-server 18.4.5.7;

}

...

ncs% commit

Figure 8: Configuring two Devices in one Transaction

vendors in case of non-standard modules. NCS allowsfor device abstractions, where you can provide a genericYANG module across vendor-specific ones.

Every commit in the scenarios described above re-sulted in a transaction across the involved devices. In thiscase the devices support the confirmed-commit capabil-ity. This means that the manager performs a commit tothe device with a time-out. If the device does not get theconfirming commit within the time-out period it revertsto the previous configuration. This is also true for restartsor if the SSH connection closes.

6

3.4 The Service Manager

In our example we have defined a service model cor-responding to web-sites and the corresponding servicelogic that maps the service model to load balancers andweb servers. The auto-rendered Web UI let operatorscreate a web site like the one illustrated in Figure 9.

Figure 9: Instantiating a Web-site Service

A fundamental part of the Service Manager is that weuse YANG to model services as well as devices. In thisway we can ensure that the service model is consistentwith the device model. We do this at compile time bychecking the YANG service model references to the de-vice model elements. At run-time, the service modelconstraints can validate elements in the device-model in-cluding referential integrity of any references. Let’s il-lustrate this with a simple example. Figure 10 shows atype-safe reference from the web-site service model tothe devices. The YANG leafref construct refers to apath in the model. The path is verified to be correct ac-cording to the model at compile time. At run-time, ifsomeone tries to delete a managed device that is referredto by a service this would violate referential integrity andNCS would reject the operation.

This service provisioning request initiates a hierarchi-cal transaction where the service instance is a parenttransaction which fires off child transactions for every

leaf lb {

description "The load balancer to use.";

mandatory true;

type leafref {

path "/ncs:ncs/ncs:managed-device/ncs:name";

}

}

Figure 10: Service-Model Reference to Device-Model

device. In this specific case the selected profile usesall web servers at the device layer. Either the completetransaction succeeds or nothing will happen. As a resultthe transaction manager stores the resulting device con-figurations in CDB as shown in Figure 11.

Figure 11: The relationship from a Service to the ActualDevice Configurations.

You see that the web-site for acme created a listeneron the load balancer with backends that maps to the ac-tual web servers. The service also created a listener onthe web server. You might wonder why there is a minus-sign for the diff. The reason is that we are actually stor-ing how to delete the service. This means that there willnever be any stale configurations in the network. As soonas you delete a service, NCS will automatically clean up.

7

4 Evaluation

4.1 Performance EvaluationWe have evaluated the performance of the solution us-ing 2000 devices in an Amazon Cloud. The Server is a4 Core CPU, 4 GB RAM, 1.0 GHz, Ubuntu 10.10 Ma-chine. Here we illustrate 4 test-cases. All test-cases areperformed as one single transaction:

1. Start the system with an empty database and uploadthe configuration over NETCONF from all devices(Figure 12 A).

2. Check if the configuration database is in sync withall the devices (Figure 12 B).

3. Perform a configuration change on all devices (Fig-ure 13 A).

4. Create 500 services instances that touch 2 deviceseach (Figure 13 B).

5. In Figure 14 we show the memory and databasejournal disc space for configuring 500 service in-stances.

All of the test-cases involve the complete transactionincluding the NETCONF round-trip to the actual devicesin the cloud. So, cold-starting NCS and uploading theconfiguration from 500 devices takes about 8 minutes(Figure 12) and 2000 devices takes about 25 minutes.The configuration synchronization check utilizes a trans-action ID to compare the last performed change fromNCS to any local changes made to the device. This testassumes that there is some way to get a transaction IDor checksum from the device that corresponds to the lastchange irrespective of which interface is used. If that isnot available and you had to get the complete configura-tion, then the numbers would be higher.

Updating the config on 500 devices takes roughly oneminute, (Figure 8). As seen by Figure 14 the in-memorydatabase has a small footprint even for large networks.In this scenario it is important to note that we always diffthe configuration change within NCS before sending itto the device. This means that we only send the actualchanges that are needed and this database comparison isincluded in the numbers. This is an area where we haveseen performance bottlenecks in previous solutions whentraditional database technologies are used.

These performance tests cover two aspects: perfor-mance of NETCONF, and our actual implementation.

NETCONF as a protocol ensures that we achieve atleast equal performance to CLI screen scraped solutionsand superior performance to SNMP based configurationsolutions. XML processing is considerably less CPU in-tensive than SSH processing.

When running a transaction that touches many man-aged devices, we use two tricks that affect performance.We pipeline NETCONF RPCs, sending several RPCs ina row, and collecting all the replies in a row. We can also(in parallel) send the requests to all participating man-aged devices, and then (in parallel) harvest the pipelinedreplies.

NCS is implemented in Erlang [3, 11] and OTP (OpenTelecom Platform) [22] which have excellent support forconcurrency and multi-core processors. A lot of efforthas gone into parallelizing the southbound requests. Forexample initial NETCONF SSH connection establish-ment is done in parallel, greatly enhancing performance.

The network configuration data is kept in a RAMdatabase together with a disk journaling component. Ifthe network is huge, the amount of RAM required can besubstantial. When the YANG files are compiled we hashall the symbols in the data models, thus the database isactually a large tree of integers. This increases process-ing speed and decreases memory footprint of the configu-ration daemon. The RAM database itself is implementedas an Erlang driver that uses skip lists [17].

Our measurements show that we can handle thousandsof devices and hundred thousands of services on off-the-shelf hardware, (4 Core CPU, 4 GB RAM, 1.0 GHz).

We have also made some measurements comparingSNMP and NETCONF performance. We read the in-terface table using SNMP get-bulk and NETCONF get.In general NETCONF performed 3 times quicker thanSNMP. The same kind of performance improvements us-ing NETCONF rather than SNMP can be found in thework by Yu and Ajarmeh [25].

4.2 NETCONF/YANG Evaluation

Let’s look at the requirements set forth by RFC 3535 andvalidate these based on our implementation.

4.2.1 Distinction between configuration data, anddata that describes operational state andstatistics

This requirement is fulfilled by YANG and NETCONF inthat you can explicitly request to get only the configura-tion data from the device, and elements in YANG are an-notated if they are configuration data or not. This greatlysimplifies the procedure to read and synchronize config-uration data from the devices to a network managementsystem. In our case, NCS can easily synchronize its con-figuration database with the actual devices.

8

Figure 12: Starting NCS and Reading the Configuration from all Devices(Dotted line represents Wall Clock Time, Solid Line CPU Time).

Figure 13: Making Device Configurations and Service Configurations

4.2.2 It is necessary to enable operators to concen-trate on the configuration of the network as awhole rather than individual devices

We have validated this from two perspectives

1. Configuring a set of devices as one transaction.

2. Transforming a service configuration to the corre-sponding device configurations.

Using NCS, we can apply configurations to a groupof devices and the transactional capabilities of NET-CONF will make sure that the whole transaction is ap-plied or no changes are made at all. The NETCONFconfirmed-commit operation has proven to be espe-cially useful in order to resolve failure scenarios. Aproblem scenario in network configuration is that de-vices may become unreachable after a reconfiguration.The confirmed-commit operation requests the deviceto take the new configuration live but if an acknowledge-

9

Figure 14: Memory and Journaling Disc Space

ment is not received within a time-out the device auto-matically rolls-back. This is the way NCS manages toroll-back configurations if one or several of the devicesin a transaction does not accept the configuration. It isnotable to see the lack of complex state-machines in NCSto do roll-backs and avoid multiple failure scenarios.

In some cases, you would like to apply a global con-figuration change to all your devices in the network. Inthe general case the transaction would fail if one of thedevices was not reachable. There is an option in NCSto backlog unresponsive devices. In this case NCS willmake the transaction succeed and store outstanding re-quests for later execution.

4.2.3 It must be easy to do consistency checks ofconfigurations.

Models in YANG contain ‘‘must’’ expressions that putconstraints on the configuration data. See for exampleFigure 3 where the must expression makes sure that theMTU is set to correct size. So for example, a NETCONFmanager can edit the candidate configuration in a deviceand ask the device to validate it. In NCS we also useYANG to specify service models. In this way we canuse must expressions to make sure that a service config-uration is consistent including the participating devices.Figure 15 shows a service configuration expression thatverifies that the subnet only exists once in the VPN.

4.2.4 It is highly desirable that text processing tools[...] can be used to process configurations.

Since NETCONF operations use well-defined XML pay-loads, it is easy to process configurations. For example

must "count(

../../mv:access-link[subnet =

current()/../subnet]) = 1" {

error-message "Subnet must be unique

within the VPN";

}

Figure 15: Service Configuration Consistency

doing a diff between the configuration in the device ver-sus the desired configuration in the management system.The CLI output in Figure 16 shows a diff between a de-vice configuration and the NCS Configuration Database.In this case a system administrator has used local toolson web server 1 and changed the document root, and re-moved the listener.

4.2.5 It is important to distinguish between the dis-tribution of configurations and the activationof a particular configuration.

The concept of multiple data-stores in NETCONF letsmanagers push the configuration to a candidate database,validate it, and then activate the configuration by com-mitting it to the running datastore. Figure 17 shows anextract from the NCS trace when activating a new con-figuration in web server 2.

10

ncs> request ncs managed-device \

www1 compare-config outformat cli <RET>

diff

ncs {

managed-device www1 {

config {

wsConfig {

global {

- ServerRoot /etc/doc;

+ ServerRoot /etc/docroot;

}

- listener 192.168.0.9 8008 {

- }

}

}

}

}

Figure 16: Comparing Configurations


www2 config wsConfig global ServerRoot /etc/doc <RET>

ncs% commit | details <RET>

ncs: SSH Connecting to admin@www2

ncs: Device: www2 Sending edit-config

ncs: Device: www2 Send commit

Commit complete.

Figure 17: Separation of Distribution of Configurationsand Activation

5 Related Work

5.1 Mapping to Taxonomy of Configura-tion Management Tools

We can map our solution to other Configuration Manage-ment solutions based on the taxonomy defined by Delaetand Joosen [7]. They define a taxonomy based on 4 cri-teria: abstraction level, specification language, consis-tency, and distributed management.

The abstraction level ranges from high-level end-to-end requirements to low-level bit-requirements. Asshown in Figure 18 and described below, in our solutionwe work with level 1-5 of the 6 mentioned abstractionlevels.

1. End-to-end Requirements - The service models inthe Service Manager expresses end-to-end require-ments including constraints expressed as XPATHmust expressions. In the case of our web site provi-sioning example this corresponds to the model for aweb site - website.yang.

Service Manager :• website.yang

Device Manager• hostsettings.yang• loadbalancer.yang• webserver.yang

NETCONFXML payload

Service Logic• web-site.java

Load BalancerWeb Server

1. End-to-end requirements

2. Instance distribution rules

3. Instance configurations

4. Implementation dependent instances

5. Configuration files

Figure 18: NCS in the Configuration Taxonomy Definedby Delaet and Joosen

2. Instance Distribution Rules - How an end-to-endservice is allocated to resources is expressed in theJava Service Logic Layer. In this layer we map theprovisioning of a web site to the corresponding loadbalancer and web-server models.

3. Instance Configurations - The changed configura-tion of devices in the Device Manager. The resultof the previous point is a diff, configuration change,sent to NCS Device Manager. The Device Man-ager has two layers. The device independent layerthat can abstract different data-models for the samefeature and the concrete device model layer. Thislayer may be vendor-independent. In Figure 18 weindicate a vendor-independent hostsetting.yangmodel which contains a unified model for host set-tings like DNS and NTP.

4. Implementation Dependent Instances - The con-crete device configuration in the NCS Device Man-ager. This is the actual configuration that is sent tothe devices in order to achieve the service instanti-ation. In the specific example of a web site it is theconfiguration change to the load balancers and webservers.

5. Configuration Files - The NETCONF XML,editconfig, payload sent to the devices. Notehowever whereas most tools work with configura-tion files, NETCONF does fine-grained configura-tion changes.

6. Bit-Configurations - Disk images are not directlymanaged by NETCONF as such.

When it comes to the specification language wehave a uniform approach based on YANG at all lev-

11

els. Delaet and Joosen characterize the specification lan-guage from four perspectives: language-based or user-interface-based, domain coverage, grouping mechanismand multi-level specification. We will elaborate on theseperspectives below.

We certainly focus on a language-based approachwhich can render various interface representations.Users can edit the configuration using the auto-renderedCLI and Web UI. You can also feed NCS with the NET-CONF XML encoding of the YANG models. NCS is ageneral purpose solution in that the domain is definedby the YANG models and not the system itself. YANGsupports groupings at the modeling level and NCS sup-ports groupings of instance configurations as config-urable templates. Templates can be applied to groups ofdevices.

NCS supports multi-level specifications which in De-laets and Joosens taxonomy refers to the ability to trans-form the configuration specifications to other formats.In our case, we are actually able to render Cisco CLIcommands automatically from the configuration change.This is a topic of its own and will not be fully coveredhere. However NCS supports YANG model-driven CLIengines that can be fed with a YANG data-model andthe engine is capable of rendering the corresponding CLIcommands.

Consistency has three perspectives in the taxonomy:dependency modeling, conflict management, and work-flow management. We do not cover workflow manage-ment. We consider workflow systems to be a client toNCS. NCS manages dependencies and conflicts basedon constraints in the models and runtime policies. Themodel constraints specifiy dependencies and rules thatare constrained by the model itself while policies are run-time constrained defined by system administrators. Weuse XPATH [6] expressions in both contexts.

Regarding conflict management NCS will detect con-flicts as violations to policies as described above. Theresult is an error message when the user tries to committhe conflicting configuration.

The final component of the taxonomy covers the as-pect of distribution. NCS supports a fine-grained AAAsystem that lets different users and client systems per-form different tasks. The agent is a NETCONF client onthe managed devices. The NCS server itself is central-ized. The primary reason here is to enable quick valida-tion of cross-device policy validation. The performanceis guaranteed by the in-memory database.

5.2 Comparison to other major configura-tion management tools

There are many well-designed configuration manage-ment tools like: CFengine [5], Puppet [15], LCFG [2]

and Bcfg2 [8]. These tools are more focused on systemand host configuration whereas we focus mostly on net-work devices and network services. This is mostly de-termined by the overall approach taken for configurationmanagement. In our model the management system has adata-model that represents the device and service config-uration. Administrators and client programs express animperative desired change based on the data-model. NCSmanages the overall transaction by the concept of a can-didate and running database which is a well-establishedprinciple for network devices.

Many host-management uses concepts of centralizedversioned configuration files rather than a database withroll-back files. Also in a host environment you can putyour specific agents on the hosts which is not the case fornetwork devices. Therefore a protocol based approachlike NETCONF/YANG is needed.

Another difference is the concept of desired state. Forhost configuration it is important to make sure that thehosts follow a centrally defined configuration which isfairly long-lived. In our case we are more focused ondoing fine-grained real-time changes based on require-ments for new services. There is room for combinationof the two approaches where host-based approaches fo-cused on configuration files address the more static setupof the device and our approach on top if that addressesdynamic changes.

It is also worth-while noting that most of the existingtools have made up their own specific languages to de-scribe configuration. YANG is a viable options for theabove mentioned tools to change to a standardized lan-guage.

There is of course a whole range of commercial tools,like Telcordia Activator [21], HP Service Activator [12],Amdocs [1], that address network and service configu-ration. While they are successfully being used for ser-vice configuration, the underlying challenges of cost andrelease-cycles for device adapters and flexibility of ser-vice models can be a challenge.

6 Conclusion and Future Work

6.1 ConclusionWe have shown that a standards-based approach for net-work configuration based on NETCONF and YANG canease the configuration management scenarios for opera-tors. Also the richness of YANG as a configuration de-scription language lends itself to automating not only thedevice communication but also the rendering of inter-faces like Command Line Interfaces and Web User In-terfaces. Much of the value in this IETF standard lies inthe transaction-based approach to configuration manage-ment and a rich domain-specific language to describe the

12

configuration and operational data. We used Erlang andin-memory database technology for our reference imple-mentation. These two choices provide performance forparallel configuration requests and fast validation of con-figuration constraints.

6.2 Future WorkWe have started to work on a NETCONF SNMP adap-tation solution which is critical to migrate from cur-rent implementations. This will allow for two scenar-ios: read-only and read-write. The read-only view is adirect mapping of SNMP MIBs to corresponding NET-CONF/YANG view, this mapping is being standardizedby IETF [13]. The read-write view is more complexand cannot be fully automated. The main reason is thatthe transactional capabilities and dependencies betweenMIB variables are not formally defined in the SNMPSMI, for example it is common that you need to setone variable before changing others. We are working oncatching the most common scenarios and define YANGextensions for those in order to automatically render asmuch as possible.

Furthermore we are working on a solution where wecan have hierarchical NCS systems in order to cover hugenetworks like nation-wide Radio Access Networks. Wewill base this on partitioning of the instantiated modelinto separate CDBs. NCS will then proxy any NET-CONF requests to the corresponding NCS system.

We are also working on two interesting features in or-der to understand the service configuration versus the de-vice configuration: “dry-run” and “service check-sync”.Committing a service activation request with dry-run cal-culates the resulting configuration changes to the devicesand displays the diff without committing it. This is help-ful in a what-if scenario: “If I provision this VPN, whathappens to my devices?”. The service check-sync fea-ture will compare a service instance with the actual con-figuration that is on devices and display any conflictingconfigurations. This is useful to detect and analyze ifand how the device configurations have been changed byany local tools in a way that breaks the service configu-rations.

References[1] AMDOCS. Amdocs service fulfillment, 2011.

http://www.amdocs.com/Products/OSS/Pages/Service-Fulfillment.aspx.

[2] ANDERSON, P., SCOBIE, A., ET AL. Lcfg: The next generation.In UKUUG Winter conference (2002), Citeseer.

[3] ARMSTRONG, J., VIRDING, R., WIKSTROM, C., ANDWILLIAMS, M. Concurrent Programming in ERLANG, 1993.

[4] BJORKLUND, M. YANG - A Data Modeling Language for theNetwork Configuration Protocol (NETCONF). RFC 6020 (Pro-posed Standard), Oct. 2010.

[5] BURGESS, M. A tiny overview of cfengine: Convergent mainte-nance agent. In Proceedings of the 1st International Workshop onMulti-Agent and Robotic Systems, MARS/ICINCO (2005), Cite-seer.

[6] CLARK, J., DEROSE, S., ET AL. XML path language (XPath)version 1.0. W3C recommendation (1999).

[7] DELAET, T., AND JOOSEN, W. Survey of configuration manage-ment tools. Katholieke Universiteit Leuven, Tech. Rep (2007).

[8] DESAI, N., LUSK, A., BRADSHAW, R., AND EVARD, R. Bcfg:A configuration management tool for heterogeneous environ-ments. Cluster Computing, IEEE International Conference on0 (2003), 500.

[9] ENCK, W., MCDANIEL, P., SEN, S., SEBOS, P., SPOEREL, S.,GREENBERG, A., RAO, S., AND AIELLO, W. Configurationmanagement at massive scale: System design and experience. InProc. of the 2007 USENIX: 21st Large Installation System Ad-ministration Conference (LISA ’07) (2007), pp. 73–86.

[10] ENNS, R. NETCONF Configuration Protocol. RFC 4741 (Pro-posed Standard), Dec. 2006.

[11] ERLANG.ORG. The erlang programming langugage, 2011.http://www.erlang.org/.

[12] HP. HP Service Activator, 2011.http://h20208.www2.hp.com/cms/solutions/ngoss/fulfillment/hpsa-suite/index.html.

[13] J. SCHONWALDER. Translation of SMIv2 MIB Mod-ules to YANG Modules. Internet-Draft, July 2011.http://tools.ietf.org/html/draft-ietf-netmod-smi-yang-01.

[14] JUNIPER. Junos XML Management Protocol, 2011.http://www.juniper.net/support/products/junoscript/.

[15] KANIES, L. Puppet: Next-generation configuration manage-ment.; login: the USENIX Association newsletter, 31 (1), 2006.

[16] MOBERG, C. A 30 Minute Introduction To NETCONF andYANG, 2011. http://www.slideshare.net/cmoberg/a-30minute-introduction-to-netconf-and-yang.

[17] PUGH, W. Skip lists: a probabilistic alternative to balanced trees.Commun. ACM 33 (June 1990), 668–676.

[18] SCHOENWAELDER, J. Overview of the 2002 IAB Network Man-agement Workshop. RFC 3535 (Informational), May 2003.

[19] SCHONWALDER, J., BJORKLUND, M., AND SHAFER, P. Net-work configuration management using NETCONF and YANG.Communications Magazine, IEEE 48, 9 (sept. 2010), 166 –173.

[20] SCOTT, M., AND BJORKLUND, M. YANG Module for NET-CONF Monitoring. RFC 6022 (Proposed Standard), Oct. 2010.

[21] TELCORDIA. Telcordia activator, 2011.http://www.telcordia.com/products/activator/index.html.

[22] TORSTENDAHL, S. Open telecom platform. Ericsson Re-view(English Edition) 74, 1 (1997), 14–23.

[23] TRAN, H., TUMAR, I., AND SCHONWALDER, J. Netconf in-teroperability testing. In Scalability of Networks and Services,R. Sadre and A. Pras, Eds., vol. 5637 of Lecture Notes in Com-puter Science. Springer Berlin / Heidelberg, 2009, pp. 83–94.10.1007/978-3-642-02627-0-7.

[24] XU, H., AND XIAO, D. Considerations on NETCONF-BasedData Modeling. In Proceedings of the 11th Asia-Pacific Sympo-sium on Network Operations and Management: Challenges forNext Generation Network Operations and Service Management(2008), Springer, p. 176.

[25] YU, J., AND AL AJARMEH, I. An Empirical Study of the NET-CONF Protocol. In Networking and Services (ICNS), 2010 SixthInternational Conference on (march 2010), pp. 253 –258.

13

automating network and service conﬁguration using netconf and...

Documents