boxes in a hurry - openvibeopenvibe.inria.fr/.../uploads/2014/10/hacking_boxes.pdfj.t. lindgren:...

1

J. T. Lindgren / OpenViBE 1

Jussi T. Lindgren, PhD [email protected]

HYBRID / Inria Rennes

HACKING

BOXES

IN A HURRY

mailto:[email protected]



2

J. T. Lindgren / OpenViBE 2 J.T. Lindgren: Hacking OpenViBE boxes Sept 2014 2

Boxes?

Making a prototype box

C++ boxes, anyone?

Inside your C++ box

Everything is not a box

Contents

3


Boxes?

4


Recall: signal processing chains --

It’s made of boxes

Signal processing boxes

offered by OpenViBE...

...potentially

allow you to do

1001 things already

OpenViBE Designer

5


• Try to use existing boxes in a clever way

• Make a wish

• Hire an engineer

• Custom order from a company...

• Do a new box yourself (this talk)

But you want to do thing #1002...

6


Input(s)

Output(s)

What is a box

Box Streams have types

(signal, stimuli, etc...)

Box::process()

{

// get input

// do things

// put output

}

Signal processing happens here

(in C++)

7


• 1. Prototype box with Matlab or Python wrapper

Or, in C++

• 2. Modify an existing box

• 3. Use Skeleton Generator to get a bare-bones box

Three ways to a box

8


Making a prototype box

9


• Use existing wrapper boxes

• Pass data between OpenViBE Matlab or Python

• Actual processing by custom .py or .m script

• Box::process() calls script for each chunk of data

• Pros & cons • + fast to prototype, no compilation or restart needed

• + no need for compiler

• - depends on matlab or python

• - potential issues if matlab/python versions change

Prototyping with Scripting Boxes

10


• Make three scripts: initialize.m, process.m, uninitialize.m

• Init/uninit called at the start/end of the playback

• Example of process.m:

• box_in is data from Designer, box_out goes to Designer

Prototyping: Matlab scripting box

function box_out = process(box_in)

disp(’process called.')

% do things to box_in

box_out = box_in;

end

11


Prototyping: Matlab scripting box, contd.

% do things to box_in

% we iterate over the pending chunks on input 1 (here SIGNAL)

for i = 1: OV_getNbPendingInputChunk(box_in,1)

% pop the first chunk to be processed, box_in used as output

[box_in, start_time, end_time, matrix_data] =

OV_popInputBuffer(box_in,1);

% signal is in matrix_data, [channels X samples] orientation

% …

% replicate header

box_in.outputs{1}.header = box_in.inputs{1}.header;

% The chunk dates are the same as the input chunk

box_in = OV_addOutputBuffer(box_in,1,start_time,end_time,matrix_data);

end

box_out = box_in;

12


• Examples & tutorials: • box-tutorials/python/

• box-tutorials/matlab/

• http://openvibe.inria.fr/tutorial-using-python-with-openvibe/

• http://openvibe.inria.fr/tutorial-using-matlab-with-openvibe/

Prototyping: more info

http://openvibe.inria.fr/tutorial-using-python-with-openvibe/









http://openvibe.inria.fr/tutorial-using-matlab-with-openvibe/









13


C++ boxes, anyone?

14


• Pros & cons • + C++ is the implementation language of OpenViBE

• + no external dependencies needed

• + code can be as fast as you can make it

• + no added python/matlab compatibility issues

• - needs a compiler (free ones are ok)

• - modifications require compilation and restart

• - C++ prototyping can be slow

• Getting started with C++

http://openvibe.inria.fr/build-instructions

C++ boxes






15


Simple:

• Decide an old box that is close to the idea of your new box

• Locate its source code from the source tree (’plugins/’)

• Hack away

Your starting point:

• Has some ’interface glue’ already in place and works

• May not have everything you need

• May have things you don’t need

A. Modifying an existing C++ box

16


- A box needs 30+ lines of ’glue code’

- More for boxes having parameters, inputs, outputs, etc

- Skeleton Generator can make the basic box code

- You get a .cpp/.h pair

- Add your custom processing code to - ::initialize() { // ... }

- ::process() { // ... }

- ::uninitialize() { // ...}

B. Skeleton Generator tool

17


Tabs to specify

various properties

of your box

When done:

Check, then

Generate!

18


• Step-by-step tutorial

http://openvibe.inria.fr/tutorial-1-implementing-a-signal-

processing-box

Time to hack into the box...?

B. Skeleton Generator tool

http://openvibe.inria.fr/tutorial-1-implementing-a-signal-processing-box













19


Inside your C++ box

20


• Each box needs to be described with e.g.

• Box identifier (random 64bit number)

• Box name

• Author

• Inputs & outputs

• Box parameters

• ...

• These are in the box .h file

• The header may also define a ’listener’ : code that reacts to box

parameter changes when made by the user

Box description

21


Box settings

Settings registered in the .h

get a GUI automatically

Save & load to scenario xml

also automatic

Getting value of an uint64 param at slot 3 (index from 0):

uint64 order

= FSettingValueAutoCast(*this->getBoxAlgorithmContext(),2);

22


- Data needs to be encoded to travel between boxes

- Codecs capsulate the data in box input/output

- Each stream needs its own codec

Codecs

Box Encoders

Decoders

In from kernel

Out to kernel

23


Codec example: initializing

myBox::initialize() {

// TSignalDecoder< myBox > m_signalDecoder; // in .h

// TSignalEncoder< myBox > m_signalEncoder; // in .h

m_signalDecoder.initialize(*this);

m_signalEncoder.initialize(*this);

// make decoder and encoder share buffer & sampling rate

m_signalEncoder.getInputMatrix().

setReferenceTarget(m_signalDecoder.getOutputMatrix());

m_signalEncoder.getInputSamplingRate().

setReferenceTarget(

m_signalDecoder.getOutputSamplingRate());

}

24


Codec example: processing

myBox::process() {

for(uint32 i=0; i<nChunks; i++) {

// decode the chunk i on input 0

m_signalDecoder.decode(0,i);

if(m_signalDecoder.isBufferReceived()) {

IMatrix* l_pMatrix = m_signalDecoder.getOutputMatrix();

// do stuff with the data in l_pMatrix, [chns x samples]

// ...

// send out

m_signalEncoder.encodeBuffer(0);

l_rDynamicBoxContext.markOutputAsReadyToSend(0,

l_rDynamicBoxContext.getInputChunkStartTime(0, i),

l_rDynamicBoxContext.getInputChunkEndTime(0, i));

}

}

25


Everything is not a box

26


• Device driver is not a box • Put it in Acquisition Server instead

• Algorithm is not a box • If a routine is shared by several boxes, consider it an ’Algorithm’

• Stimulators may not want to be boxes • Should be external apps outside Designer

• Send their triggers to Acquisition Server

• For timing reasons, avoid making stimulators as boxes

27


Here we have skipped a lot of details

Box coding is best learned by doing

More about at

http://openvibe.inria.fr

That’s it!

http://openvibe.inria.fr/

http://openvibe.inria.fr/

boxes in a hurry - openvibeopenvibe.inria.fr/.../uploads/2014/10/hacking_boxes.pdfj.t. lindgren:...

Documents