implementing network protocols in the - schd.wsschd.ws/hosted_files/openiotelcna2017/8d/implementing...
TRANSCRIPT
Leandro A. F. [email protected]
Implementing Network Protocols in the Zephyr* Project
OpenIoT Summit 2017 — Portland, OR
Implementing network protocols in Zephyr* / OpenIoT 2017
About me
Implementing network protocols in Zephyr* / OpenIoT 2017
Zephyr* in one slide● Microcontroller operating system● Very small footprint
○ Will run in 8KiB of RAM
● Open Source under Apache* 2.0 license● Hosted by Linux* Foundation● Supports multiple architectures
Check the booths for demos and more info!
Implementing network protocols in Zephyr* / OpenIoT 2017
Table of Contents● New IP stack in Zephyr*● Currently implemented network protocols● Patterns
Implementing network protocols in Zephyr* / OpenIoT 2017
Zephyr* has a new IP stackOld stack
● uIP from Contiki*○ No simultaneous IPv6 and IPv4○ Only one network interface
● Ported to Zephyr○ Didn't fit well with the rest of the system○ Parts of Contiki had to be ported over○ Changes made it unstable, "unfixable"
New stack
● Multiple interfaces● Simultaneous IPv6 and IPv4● Better fits the rest of the system
○ Buffer management● Available since 1.6
○ 1.7 will enable TCP
Implementing network protocols in Zephyr* / OpenIoT 2017
Multiple Interfaces● Configurable at compile time● Ethernet, Bluetooth®, 802.15.4, SLIP (for testing)
○ Soon: Wi-Fi*
● Can mix and match interface types
Implementing network protocols in Zephyr* / OpenIoT 2017
Simultaneous IPv4 and IPv6
IPv4 IPv6 Code size* (.text)
✘ ✓ ~11KiB
✓ ✘ ~9KiB
✓ ✓ ~12KiB
Non-scientific measurement, in KiB, of the echoserver sample application in multiple configurations
* Measurements may vary depending on target architecture and featureset
Implementing network protocols in Zephyr* / OpenIoT 2017
Buffer Management● Zephyr* is "heapless" inside the kernel
○ Memory pools used instead○ Separate pools for transmitting, receiving, and data○ Timeouts for obtaining things from pools
● Fragment chains○ Fragments of fixed size (128 bytes each)○ Chain can be compacted or copied to a linear buffer
Implementing network protocols in Zephyr* / OpenIoT 2017
Currently ImplementedNetwork Protocols
Implementing network protocols in Zephyr* / OpenIoT 2017
DNS Domain Name System● Both IPv4 and IPv6 queries are supported
○ A and AAAA queries only
● Support for concurrent queries○ Pool of 512 byte chunks (maximum UDP response size as per RFC1035)○ Configurable number of concurrent queries
● Fragment chain from the network copied to a linear buffer● Implemented from scratch● Available as part of the platform
Implementing network protocols in Zephyr* / OpenIoT 2017
HTTP Hypertext Transport Protocol● Both client and server● HTTP/1.0 and HTTP/1.1● Uses parser from Nginx● Available as sample applications (client/server)
Implementing network protocols in Zephyr* / OpenIoT 2017
NATS publisher/subscriber protocol● Asynchronous API● Line-based protocol● Implemented from scratch● Available as a sample application
Implementing network protocols in Zephyr* / OpenIoT 2017
MQTT Message Queue Telemetry Transport● Asynchronous API● Implemented from scratch● Available as part of the platform
Implementing network protocols in Zephyr* / OpenIoT 2017
CoAP Constrained Application Protocol● Asynchronous API
○ GET, POST, PUT, and DELETE methods
● Resource observation & notifications● Implementation based on lessons learned from the Soletta™ project● Available as part of the platform
Implementing network protocols in Zephyr* / OpenIoT 2017
IRC Internet Relay Chat● Asynchronous API● Line-based protocol● Implemented from scratch● Available as a sample application
Implementing network protocols in Zephyr* / OpenIoT 2017
Patterns
Implementing network protocols in Zephyr* / OpenIoT 2017
Asynchronous APIs● Used with request-less protocols● Information can be received at any time
○ No clients blocked while reading from the network○ Background work
Implementing network protocols in Zephyr* / OpenIoT 2017
Buffer linearization● Used to parse structs larger than the fragment size
○ Fragments usually 128 bytes
● Not optimal○ Unnecessary memory copies○ Memory pools for linearized buffers○ May go away in the future
Implementing network protocols in Zephyr* / OpenIoT 2017
Reusing vs. Re-implementing● Implementations may be unsuitable for
embedded systems○ Dynamic memory allocations○ Assumes a POSIX* operating system○ Might depend on heavyweight libraries
(libstdc++, glib, etc)○ Etc.
● Existing implementations may be mature
● Reimplementing is tricky○ Not leveraging possible years of
development● A lot of control● Tighter integration with the rest of the
system
Implementing network protocols in Zephyr* / OpenIoT 2017
Sample application vs. Platform APISample application
● Testbed for APIs● No guarantee the APIs will be stable● Often only suitable for demonstration
purposes● Not necessarily part of the "IoT Suite"
Available as part of the platform
● Stable, complete APIs● Suitable for production use● Protocols from the "IoT Suite"
Implementing network protocols in Zephyr* / OpenIoT 2017
Compromises: example● NATS protocol requires a JSON parser● Most JSON parsers require heap memory● Wrote our own:
○ Zero-copy○ Decodes directly to a struct○ "Type safe"○ No temporary structs
● Only necessary things implemented:○ No nested objects/arrays○ UTF-8 aware but not strict○ No floating point numbers○ Up to 32 fields
Implementing network protocols in Zephyr* / OpenIoT 2017
Code Samples
Implementing network protocols in Zephyr* / OpenIoT 2017
Asynchronous APIs
static const char *const led_default_path[] = { "led", NULL };static struct zoap_resource resources[] = { ZOAP_WELL_KNOWN_CORE_RESOURCE , { .get = led_get, .post = led_post, .put = led_put, .path = led_default_path, .user_data = /* … */, }, { .get = dummy_get, .path = dummy_path, .user_data = /* … */, }, { },};
The CoAP implementation will call the handler functions for each method
Implementing network protocols in Zephyr* / OpenIoT 2017
Asynchronous APIs
static const char *const led_default_path[] = { "led", NULL };static struct zoap_resource resources[] = { ZOAP_WELL_KNOWN_CORE_RESOURCE , { .get = led_get, .post = led_post, .put = led_put, .path = led_default_path, .user_data = /* … */, }, { .get = dummy_get, .path = dummy_path, .user_data = /* … */, }, { },};
Users of the API don't need to worry about matching resource paths
Implementing network protocols in Zephyr* / OpenIoT 2017
JSON parsing
struct my_struct { int number; char *string;};
#define FIELD(struct_, member_, type_) /* … */
static const struct json_obj_descr my_struct_descr[] = { FIELD(struct my_struct, number, JSON_TOK_NUMBER), FIELD(struct my_struct, string, JSON_TOK_STRING),};
struct my_struct s;int32_t result = json_obj_parse(json_data, json_data_len,
my_struct_descr, ARRAY_SIZE(my_struct_descr), &s);
Implementing network protocols in Zephyr* / OpenIoT 2017
JSON parsing
struct my_struct { int number; char *string;};
#define FIELD(struct_, member_, type_) /* … */
static const struct json_obj_descr my_struct_descr[] = { FIELD(struct my_struct, number, JSON_TOK_NUMBER), FIELD(struct my_struct, string, JSON_TOK_STRING),};
struct my_struct s;int32_t result = json_obj_parse(json_data, json_data_len,
my_struct_descr, ARRAY_SIZE(my_struct_descr), &s);
The struct is described, and, while parsing, types are checked against the provided description.
Implementing network protocols in Zephyr* / OpenIoT 2017
JSON parsing
struct my_struct { int number; char *string;};
#define FIELD(struct_, member_, type_) /* … */
static const struct json_obj_descr my_struct_descr[] = { FIELD(struct my_struct, field, JSON_TOK_NUMBER), FIELD(struct my_struct, field, JSON_TOK_STRING),};
struct my_struct s;int32_t result = json_obj_parse(json_data, json_data_len,
my_struct_descr, ARRAY_SIZE(my_struct_descr), &s);
If successful, result is 0x3: both number (first field, 1<<0) and string (1<<1) fields have been decoded.
Implementing network protocols in Zephyr* / OpenIoT 2017
Wrapping up● Some of the differences between old and new IP stacks● Protocols currently implemented● Patterns used by the implementations● Some code samples
What would you do differently?
What protocols are you going to implement in Zephyr*?
Implementing network protocols in Zephyr* / OpenIoT 2017
DisclaimerIntel, the Intel logo, and Soletta are trademarks of Intel Corporation or its subsidiaries in the U.S. and/or other countries.*Other names and brands may be claimed as the property of others.© Intel Corporation