![Page 1: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/1.jpg)
Neobuffer: Cross-Process Channels in Rust
Bernardo MeurerJune 28th, 2019
Standard Cognition
![Page 2: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/2.jpg)
whoami
• Bernardo Meurer(lovesegfault)
• Systems Engineer @Standard Cognitionhttp://standard.ai
• [email protected]• http://lovesegfault.com/
1
![Page 3: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/3.jpg)
Outline
• Motivation• Requirements• Alternatives• Neobuffer
• Components• Ringbuffer• Sink• Source• Quirks• Results• Future work
• Questions
2
![Page 4: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/4.jpg)
whoarewe(?!)
• Standard Cognition• SF + Tokyo• Autonomous Checkout• Python + Rust• Many interesting problemsto solve
• We’re hiring!
3
![Page 5: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/5.jpg)
Motivation
• We deal with a many cameras.
• We do experiments very frequently.• How to multiplex the video streams?• How to unify the API for controlling distinct cameras?
4
![Page 6: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/6.jpg)
Motivation
5
![Page 7: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/7.jpg)
Motivation
• We deal with a many cameras.• We do experiments very frequently.
• How to multiplex the video streams?• How to unify the API for controlling distinct cameras?
6
![Page 8: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/8.jpg)
Motivation
7
![Page 9: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/9.jpg)
Motivation
• We deal with a many cameras.• We do experiments very frequently.• How to multiplex the video streams?
• How to unify the API for controlling distinct cameras?
8
![Page 10: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/10.jpg)
Motivation
• We deal with a many cameras.• We do experiments very frequently.• How to multiplex the video streams?• How to unify the API for controlling distinct cameras?
8
![Page 11: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/11.jpg)
Sensord/Camd
• We still haven’t quite decided on a name...
• Camera daemon• Clients use ZMQ to communicate with it• Has client libs for Rust, and Python (PyO3)• Reasonably large Rust project• A small part of a huge system• One challenge was clear from inception...
9
![Page 12: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/12.jpg)
Sensord/Camd
• We still haven’t quite decided on a name...• Camera daemon
• Clients use ZMQ to communicate with it• Has client libs for Rust, and Python (PyO3)• Reasonably large Rust project• A small part of a huge system• One challenge was clear from inception...
9
![Page 13: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/13.jpg)
Sensord/Camd
• We still haven’t quite decided on a name...• Camera daemon• Clients use ZMQ to communicate with it
• Has client libs for Rust, and Python (PyO3)• Reasonably large Rust project• A small part of a huge system• One challenge was clear from inception...
9
![Page 14: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/14.jpg)
Sensord/Camd
• We still haven’t quite decided on a name...• Camera daemon• Clients use ZMQ to communicate with it• Has client libs for Rust, and Python (PyO3)
• Reasonably large Rust project• A small part of a huge system• One challenge was clear from inception...
9
![Page 15: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/15.jpg)
Sensord/Camd
• We still haven’t quite decided on a name...• Camera daemon• Clients use ZMQ to communicate with it• Has client libs for Rust, and Python (PyO3)• Reasonably large Rust project
• A small part of a huge system• One challenge was clear from inception...
9
![Page 16: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/16.jpg)
Sensord/Camd
$ tokei ./sensord-------------------------------------------------------------------------------Language Files Lines Code Comments Blanks
-------------------------------------------------------------------------------C Header 2 4 4 0 0GLSL 2 63 54 1 8Markdown 1 29 29 0 0Nix 1 44 34 0 10Python 1 28 17 4 7Rust 47 9391 7457 880 1054Plain Text 1 13 13 0 0TOML 14 274 240 0 34
-------------------------------------------------------------------------------Total 69 9846 7848 885 1113
-------------------------------------------------------------------------------
10
![Page 17: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/17.jpg)
Sensord/Camd
• We still haven’t quite decided on a name...• Camera daemon• Clients use ZMQ to communicate with it• Has client libs for Rust, and Python (PyO3)• Reasonably large Rust project• A small part of a huge system
• One challenge was clear from inception...
11
![Page 18: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/18.jpg)
Sensord/Camd
12
![Page 19: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/19.jpg)
Sensord/Camd
• We still haven’t quite decided on a name...• Camera daemon• Clients use ZMQ to communicate with it• Has client libs for Rust, and Python (PyO3)• Reasonably large Rust project• A small part of a huge system• One challenge was clear from inception...
13
![Page 20: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/20.jpg)
Sensord/Camd
How to connect camd to the client processes?
14
![Page 21: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/21.jpg)
Channel X: Requirements
• Fast
• Lock-Free• Flexible• Cross-thread• Cross-process• Safe• Multi-consumer
15
![Page 22: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/22.jpg)
Channel X: Requirements
• Fast• Lock-Free
• Flexible• Cross-thread• Cross-process• Safe• Multi-consumer
15
![Page 23: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/23.jpg)
Channel X: Requirements
• Fast• Lock-Free• Flexible
• Cross-thread• Cross-process• Safe• Multi-consumer
15
![Page 24: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/24.jpg)
Channel X: Requirements
16
![Page 25: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/25.jpg)
Channel X: Requirements
• Fast• Lock-Free• Flexible• Cross-thread
• Cross-process• Safe• Multi-consumer
17
![Page 26: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/26.jpg)
Channel X: Requirements
• Fast• Lock-Free• Flexible• Cross-thread• Cross-process
• Safe• Multi-consumer
17
![Page 27: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/27.jpg)
Channel X: Requirements
• Fast• Lock-Free• Flexible• Cross-thread• Cross-process• Safe
• Multi-consumer
17
![Page 28: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/28.jpg)
Channel X: Requirements
• Fast• Lock-Free• Flexible• Cross-thread• Cross-process• Safe• Multi-consumer
17
![Page 29: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/29.jpg)
Channel X: Who could it be?
• stdlib channels?
• crossbeam?• rb?• ringbuf?
18
![Page 30: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/30.jpg)
Channel X: Who could it be?
• stdlib channels?• crossbeam?
• rb?• ringbuf?
18
![Page 31: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/31.jpg)
Channel X: Who could it be?
• stdlib channels?• crossbeam?• rb?
• ringbuf?
18
![Page 32: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/32.jpg)
Channel X: Who could it be?
• stdlib channels?• crossbeam?• rb?• ringbuf?
18
![Page 33: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/33.jpg)
Channel X: Neobuffer
19
![Page 34: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/34.jpg)
Neobuffer
• Written from scratch• 100% Rust• Collaboration with ekleog (Leo Gaspard) and nagisa(Simonas Kaslauskas)
• Based on discussions with eddyb and amanieu
20
![Page 35: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/35.jpg)
Neobuffer: Components
• Common traits for process-level Sync and Send• interprocess-traits — ProcSync and ProcSend• github.com/standard-ai/interprocess-traits
• Shared memory for cross-process data sharing• shmem — An ergonomic wrapper around shared memoryfor Rust
• github.com/standard-ai/shmem• Unix Domain Sockets for sending file descriptors acrossprocesses
• sendfd — Utility crate that simplifies fd sharing.• github.com/standard-ai/sendfd
• Ringbuffer for the underlying data structure• Atomics for the indexes• Careful application of memory orderings to guaranteeconsistency
• Safe wrap around• github.com/standard-ai/neobuffer
21
![Page 36: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/36.jpg)
Neobuffer: Components
• Common traits for process-level Sync and Send• interprocess-traits — ProcSync and ProcSend• github.com/standard-ai/interprocess-traits
• Shared memory for cross-process data sharing• shmem — An ergonomic wrapper around shared memoryfor Rust
• github.com/standard-ai/shmem
• Unix Domain Sockets for sending file descriptors acrossprocesses
• sendfd — Utility crate that simplifies fd sharing.• github.com/standard-ai/sendfd
• Ringbuffer for the underlying data structure• Atomics for the indexes• Careful application of memory orderings to guaranteeconsistency
• Safe wrap around• github.com/standard-ai/neobuffer
21
![Page 37: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/37.jpg)
Neobuffer: Components
• Common traits for process-level Sync and Send• interprocess-traits — ProcSync and ProcSend• github.com/standard-ai/interprocess-traits
• Shared memory for cross-process data sharing• shmem — An ergonomic wrapper around shared memoryfor Rust
• github.com/standard-ai/shmem• Unix Domain Sockets for sending file descriptors acrossprocesses
• sendfd — Utility crate that simplifies fd sharing.• github.com/standard-ai/sendfd
• Ringbuffer for the underlying data structure• Atomics for the indexes• Careful application of memory orderings to guaranteeconsistency
• Safe wrap around• github.com/standard-ai/neobuffer
21
![Page 38: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/38.jpg)
Neobuffer: Components
• Common traits for process-level Sync and Send• interprocess-traits — ProcSync and ProcSend• github.com/standard-ai/interprocess-traits
• Shared memory for cross-process data sharing• shmem — An ergonomic wrapper around shared memoryfor Rust
• github.com/standard-ai/shmem• Unix Domain Sockets for sending file descriptors acrossprocesses
• sendfd — Utility crate that simplifies fd sharing.• github.com/standard-ai/sendfd
• Ringbuffer for the underlying data structure• Atomics for the indexes• Careful application of memory orderings to guaranteeconsistency
• Safe wrap around• github.com/standard-ai/neobuffer 21
![Page 39: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/39.jpg)
Neobuffer: Ringbuffer, Sink, and Source
/// A shared-memory ring buffer that contains items of type `T`, has one writer/// and `NbReaders` readers that each read all the items the writer writes.pub struct RingBuffer<T, NbReaders> {
/// The shared memory regiondata: Shared<c_void>,
/// The maximum number of elements this ring buffer can contain at a time.size: u64,
/// Whether the one writer of this ring buffer has already been given out.writer_given: AtomicBool,
/// The number of readers that have already been given out.readers_given: AtomicUsize,
/// Phantom argument to keep type arguments alive.phantom: PhantomData<(T, NbReaders)>,
}
22
![Page 40: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/40.jpg)
Neobuffer: Ringbuffer, Sink, and Source
• Shared<T> just means T is in shared memory
• Shared<c_void> is a special case for data not sized• But what exactly is in Shared<c_void>?• M is the Ringbuffer metadata, and the magic smoke ofNeobuffer
23
![Page 41: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/41.jpg)
Neobuffer: Ringbuffer, Sink, and Source
• Shared<T> just means T is in shared memory• Shared<c_void> is a special case for data not sized
• But what exactly is in Shared<c_void>?• M is the Ringbuffer metadata, and the magic smoke ofNeobuffer
23
![Page 42: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/42.jpg)
Neobuffer: Ringbuffer, Sink, and Source
• Shared<T> just means T is in shared memory• Shared<c_void> is a special case for data not sized• But what exactly is in Shared<c_void>?
• M is the Ringbuffer metadata, and the magic smoke ofNeobuffer
23
![Page 43: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/43.jpg)
Neobuffer: Ringbuffer, Sink, and Source
24
![Page 44: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/44.jpg)
Neobuffer: Ringbuffer, Sink, and Source
• Shared<T> just means T is in shared memory• Shared<c_void> is a special case for data not sized• But what exactly is in Shared<c_void>?• M is the Ringbuffer metadata, and the magic smoke ofNeobuffer
25
![Page 45: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/45.jpg)
Neobuffer: Ringbuffer, Sink, and Source
/// The metadata for the data stored in the `Shared`.////// The actual data in the ringbuffer precedes this metadata, so that it can be/// page-aligned. Elements between the minimal element of `reader_steps` and/// `writer_step` (modulo `Size`) are the only initialized ones. The safety of/// this relies on the fact that things in a `Shared` are guaranteed to not be/// dropped, and to the dropping operations being manually handled.#[repr(C)]struct Metadata {
/// Number of readers allocated in the metadata area (following this struct).////// This is immutable.reader_count: usize,
/// The number of items that have ever been written. Monotonically/// increasing. `u64` is used here because `u32` might not be enough to/// handle a reasonable number of ring buffer additions. However, all uses/// that are bounded by `Size` can safely assume it fits in `usize`.writer_step: AtomicU64,
/// The number of outstanding references to this shared memory region.////// This knowledge is used to know when the remaining elements should be/// dropped.references: AtomicU64,
}
26
![Page 46: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/46.jpg)
Neobuffer: Ringbuffer, Sink, and Source
impl<T, R> RingBuffer<T, R>where
// Ts are not created/inserted/shared here, so T does not need any bound here.R: Unsigned + NonZero,
{pub fn new(size: usize) -> Result<RingBuffer<T, R>, shmem::Error> {...}pub fn get_sink(&self) -> Result<Sink<T>, Error> {...}pub fn get_source(&self) -> Result<Source<T>, Error> {...}
}
27
![Page 47: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/47.jpg)
Neobuffer: Ringbuffer, Sink, and Source
/// A sink that can send data into a `RingBuffer`.pub struct Sink<T> {
/// Shared pointer to the data.data: Shared<c_void>,
/// Number of `T` that `data` has been allocated with.size: u64,
/// Local copy of `data.writer_step`, that is not atomic for performance.writer_step: u64,
/// Local helper that contains a value guaranteed to be lower to or equal/// to all values in `data.reader_steps`minimum_reader_step: Cell<u64>,
/// Phantom argument to keep type arguments alive.phantom: PhantomData<T>,
}
28
![Page 48: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/48.jpg)
Neobuffer: Ringbuffer, Sink, and Source
impl<T> Sink<T> {pub fn push(&mut self, item: T) -> nb::Result<(), Error> {...}
}
29
![Page 49: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/49.jpg)
Neobuffer: Ringbuffer, Sink, and Source
/// A sink that can read data from a `RingBuffer`.pub struct Source<T> {
/// Shared pointer to the data.data: Shared<c_void>,
/// Size of the ring buffer this is linked with.size: u64,
/// Identifier of this reader in the `data.reader_steps` array.reader_id: usize,
/// Local copy of `data.reader_steps[reader_id]`, for performance.reader_step: u64,
/// Under-approximation of `data.writer_step`, local for performance.minimum_writer_step: Cell<u64>,
/// Phantom argument to keep type arguments alive.phantom: PhantomData<T>,
}
30
![Page 50: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/50.jpg)
Neobuffer: Ringbuffer, Sink, and Source
impl<T> Source<T> {pub fn available_size(&self) -> usize {...}
/// Advance the reader discarding `n` lowest numbered elements in the buffer.////// This allows elements, which have been advanced past by all readers, to/// be overwritten.pub fn advance(&mut self, n: usize) -> nb::Result<(), Infallible> {...}
/// Get a reference to `i`-th element stored in the buffer.////// Once the use of the reference is done, [`Source::advance`] must be/// called to advance the reader.pub fn get(&self, i: usize) -> nb::Result<&T, Infallible> {...}
/// Handy combination of `get()` and `advance()`pub fn pop(&mut self) -> nb::Result<T, Infallible> {...}
}
31
![Page 51: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/51.jpg)
Neobuffer: Quirks
• Consumption is local
• Relative indexing• Only works on Linux• You have to be surgically careful with atomic orderings
32
![Page 52: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/52.jpg)
Neobuffer: Quirks
• Consumption is local• Relative indexing
• Only works on Linux• You have to be surgically careful with atomic orderings
32
![Page 53: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/53.jpg)
Neobuffer: Quirks
• Consumption is local• Relative indexing• Only works on Linux
• You have to be surgically careful with atomic orderings
32
![Page 54: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/54.jpg)
Neobuffer: Quirks
• Consumption is local• Relative indexing• Only works on Linux• You have to be surgically careful with atomic orderings
32
![Page 55: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/55.jpg)
Neobuffer: Results
one_writer_many_batching_readers_big ... 10,693,940 ns/iter (+/- 931,225) = 1568 MB/sone_writer_many_batching_readers_huge ... 2,207,889 ns/iter (+/- 370,206) = 30395 MB/sone_writer_many_batching_readers_small ... 9,169,666 ns/iter (+/- 411,746) = 7 MB/sone_writer_many_readers_big ... 7,919,200 ns/iter (+/- 584,713) = 2118 MB/sone_writer_many_readers_huge ... 2,219,540 ns/iter (+/- 186,404) = 30235 MB/sone_writer_many_readers_small ... 6,706,252 ns/iter (+/- 480,979) = 9 MB/sone_writer_one_batching_reader_big ... 3,522,033 ns/iter (+/- 18,138) = 4763 MB/sone_writer_one_batching_reader_huge ... 2,188,906 ns/iter (+/- 19,540) = 30658 MB/sone_writer_one_batching_reader_small ... 3,369,997 ns/iter (+/- 314,502) = 19 MB/sone_writer_one_reader_big ... 5,138,619 ns/iter (+/- 263,518) = 3264 MB/sone_writer_one_reader_huge ... 2,196,294 ns/iter (+/- 31,967) = 30555 MB/sone_writer_one_reader_small ... 3,213,515 ns/iter (+/- 360,083) = 20 MB/sone_writer_two_batching_readers_big ... 5,827,360 ns/iter (+/- 160,344) = 2879 MB/sone_writer_two_batching_readers_huge ... 2,201,807 ns/iter (+/- 14,876) = 30478 MB/sone_writer_two_batching_readers_small ... 4,877,466 ns/iter (+/- 184,255) = 13 MB/sone_writer_two_readers_big ... 5,310,644 ns/iter (+/- 149,770) = 3159 MB/sone_writer_two_readers_huge ... 2,195,695 ns/iter (+/- 91,573) = 30563 MB/sone_writer_two_readers_small ... 4,028,670 ns/iter (+/- 403,310) = 16 MB/s
33
![Page 56: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/56.jpg)
Neobuffer: Results
Assuming 4 parallel readers and a buffer size of 8:
• 7 Mpps (1 byte packets)• 50Mpps with a buffer size of 2048
• 300 Gbps (2MB packets)
34
![Page 57: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/57.jpg)
Neobuffer: Results
Assuming 4 parallel readers and a buffer size of 8:
• 7 Mpps (1 byte packets)• 50Mpps with a buffer size of 2048
• 300 Gbps (2MB packets)
34
![Page 58: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/58.jpg)
Neobuffer: Future work
• Improve ergonomics around failure cases
• Split pushing into allocate + commit (remove acopy/clone)
• Swappable backend• MPMC• Futures• Global consumption
35
![Page 59: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/59.jpg)
Neobuffer: Future work
• Improve ergonomics around failure cases• Split pushing into allocate + commit (remove acopy/clone)
• Swappable backend• MPMC• Futures• Global consumption
35
![Page 60: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/60.jpg)
Neobuffer: Future work
• Improve ergonomics around failure cases• Split pushing into allocate + commit (remove acopy/clone)
• Swappable backend
• MPMC• Futures• Global consumption
35
![Page 61: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/61.jpg)
Neobuffer: Future work
• Improve ergonomics around failure cases• Split pushing into allocate + commit (remove acopy/clone)
• Swappable backend• MPMC
• Futures• Global consumption
35
![Page 62: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/62.jpg)
Neobuffer: Future work
• Improve ergonomics around failure cases• Split pushing into allocate + commit (remove acopy/clone)
• Swappable backend• MPMC• Futures
• Global consumption
35
![Page 63: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/63.jpg)
Neobuffer: Future work
• Improve ergonomics around failure cases• Split pushing into allocate + commit (remove acopy/clone)
• Swappable backend• MPMC• Futures• Global consumption
35
![Page 64: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/64.jpg)
Summary
• Lock-free, fast, cross-process channel
• Ringbuffer in shared memory• Public supporting libraries• Atomics to manage read/write indexes• Safe interface for cross-process communication
36
![Page 65: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/65.jpg)
Summary
• Lock-free, fast, cross-process channel• Ringbuffer in shared memory
• Public supporting libraries• Atomics to manage read/write indexes• Safe interface for cross-process communication
36
![Page 66: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/66.jpg)
Summary
• Lock-free, fast, cross-process channel• Ringbuffer in shared memory• Public supporting libraries
• Atomics to manage read/write indexes• Safe interface for cross-process communication
36
![Page 67: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/67.jpg)
Summary
• Lock-free, fast, cross-process channel• Ringbuffer in shared memory• Public supporting libraries• Atomics to manage read/write indexes
• Safe interface for cross-process communication
36
![Page 68: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/68.jpg)
Summary
• Lock-free, fast, cross-process channel• Ringbuffer in shared memory• Public supporting libraries• Atomics to manage read/write indexes• Safe interface for cross-process communication
36
![Page 69: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/69.jpg)
Questions?
There will be no questionsAre there any questions?
37
![Page 70: Neobuffer: Cross-Process Channels in Rust · Outline • Motivation • Requirements • Alternatives • Neobuffer • Components • Ringbuffer • Sink • Source • Quirks •](https://reader033.vdocuments.mx/reader033/viewer/2022060523/60534707ace0752ae7574dc7/html5/thumbnails/70.jpg)
Made with Free Software
This presentation was made using Free (as in Freedom) andOpen Source software:
• Vim• LATEX + Beamer + metropolis• Graphviz• Inkscape• Some dude on Fiverr
38