network operating system harmonization architecture and automated sanity test demo

Cisco Confidential© 2012 Cisco and/or its affiliates. All rights reserved. 1

Network Operating System Harmonization Architecture and Automated Sanity Test

DemoCisco Research Center Training

Vikram Hosakote, Sr. Software Developer, NOSTG

Cisco Systems, Boxborough MA, [email protected]

04/18/2014

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 2

OSH Manageability Test Architecture


Python show display templates

Testing for templates is initially done in internal dev env in a standalone fashion, independent of cli parser. Cli parser is not available in internal dev env.

Released sanity includes basic template testing of the entire test yang, on internal dev env. Additional tests are being added for python template api feature/functional testing, and negative python template api tests.

Initial integration with cli parser will occur on NXOS.

Target is to use same python based test framework for sanity testing and end-to-end testing with CLI parser. Test framework is being implemented to handle standalone vs. cli parser execution, & xmascon vs. rtr config mode for config data setting.


CLI parser and Nvgen

Parser/Nvgen Test plan positive testcases EDCS-1293270

Parser/Nvgen Test Plan negative testcases EDCS-1295063

TIMS test report:http://tims.cisco.com/warp.cmd?ent=Tzm1557d

Bug Scrubber:http://wwwin-ottawa.cisco.com/tfoggoa/Scrubber/showquery.html?query=akalambu-7

DE-DT weekly Sync-up

CLI sanity test script

http://tims.cisco.com/warp.cmd?ent=Tzm1557d

http://wwwin-ottawa.cisco.com/tfoggoa/Scrubber/showquery.html?query=akalambu-7


XTF test agent

Agent side APIs are tested using the test framework implemented in XTF

Agent side APIs are documented here - http://wwwin-ensoft.cisco.com/osh-mgbl/apis/xmas/latest/internal/xmas__agent_8h.html#a515d63f9bfc0301005b581e91017d738

The XTF Agent tests cover get/set all objects in test-comp.yang model, register APIs, action APIs, singleton write, bulk write, lookup, utility, bag encode and decode APIs

http://wwwin-ensoft.cisco.com/osh-mgbl/apis/xmas/latest/internal/xmas__agent_8h.html

http://wwwin-ensoft.cisco.com/osh-mgbl/apis/xmas/latest/internal/xmas__agent_8h.html


NETCONF

NETCONF is an IETF defined protocol to provide mechanisms to install, manipulate, and delete the configuration of network devices

The NETCONF server application resides on the IOS devices and accepts connections from external NETCONF client applications. The connected session then behaves as an initiator or listener

NETCONF capabilities & protocol operation feature test includes <get-config> , <edit-config>, <lock/unlock> <subtree filtering> and can be found at http://wwwin-eng.cisco.com/Eng/NMTG/Device_Instrumentation/NETCONF_sw_func_spec.doc

http://wwwin-eng.cisco.com/Eng/NMTG/Device_Instrumentation/NETCONF_sw_func_spec.doc

http://wwwin-eng.cisco.com/Eng/NMTG/Device_Instrumentation/NETCONF_sw_func_spec.doc


YANG compiler and southbound code

The test YANG and XMAS files are compiled using the OSH YANG / XMAS compiler.

Once the YANG and XMAS files are compiled, the DevEnv / Polaris / NXOS workspace is built.

The southbound code is the backend application C code that has the implementation for all the backend generated XMAS callbacks for registration, config, oper, RPC, notifications, and PCA.

YANG compiler uses pyang, Cython (Python to C translator integrated with CPython), and Jinja2


XMAS datatype translation

File.pyx Cython Compiler File.c C compiler File.so

File.py

App.py

import


OSH manageability sanities

XMAS YANG compiler sanity script

Python show display template sanity script

XTF test agent sanity script

CLI compiler sanity script

Daily sanities available in DevEnv (EnXR) and NXOS (osh_ual_72 and Jenkins) that run a total of 340 tests everyday

Info about these sanities on OSH manageability wiki - http://wikicentral.cisco.com/display/GROUP/NOSTG+OSH+UAL+Manageability+Test+Wiki

http://wikicentral.cisco.com/display/GROUP/NOSTG+OSH+UAL+Manageability+Test+Wiki




Sample OSH mgbl daily sanity email report


OSH manageability P&S testing

Below are the OSH performance and scaling numbers tested config lists with 5000 keys - mac address, uint32, int64, string oper lists with 5000 keys - mac address, uint32, int64, string 5000 leaves in config xmas:item list – mac address, uint32, int64, string 5000 leaves in xmas:item container - mac address, uint32, int64, string Send 1000 RPC results – sync and async Generate 1000 notifications

OSH manageability P&S test plan - EDCS-1380817


OSH manageability HA testing

HA switchover testing done on Polaris (ASR 1006), and NXOS (N7K SUP2)

Process restart tests done – test_comp_main, sysdb_svr, xyang_svr, xosds

The following HA tests were done: Python show templates XTF agent tests YANG XMAS compiler tests CLI tests

OSH manageability HA test plan - EDCS-1354815


OSH manageability test wiki


The OSH manageability wiki has information about OSH mgbl test plans and the OSH mgbl test strategy Bug scrubber links (XMAS, NETCONF, and xos-parser bugs) Weekly status meeting minutes TIMS links (EnXR, Polaris, and NXOS) OSH mgbl sanity scripts OSH mgbl training

OSH mgbl test mailing alias – osh-mgbl-test





Thank you.

network operating system harmonization architecture and automated sanity test demo

Engineering