Piece of cake – testing remoteembedded devices made easy

using open-hardware MuxPi

.

Paweł Wieczorek

March 12, 2018

Samsung R&D Institute Poland

Agenda.

1. Introduction

2. Previous efforts

3. Idea

4. Hardware

5. Software

6. Next steps

7. Conclusion 1/50

Introduction.

Tizen.

2/50

Use cases.

https://news.s6msung.com/glob6l/tizen- - -first-milestone-rele6se-to-open-new-opportunities-in-the-iot-er6

3/50

Release engineering.

• Continuous platform development

• QA step prior pulling new changes

• Package internal tests are not enough

4/50

Remote accessibility.

• Easy to storein a secure manner

• Less effort thanper developer

• Better utilizedwhen shared

.....

5/50

Piece of cake (with MuxPi).

6/50

Piece of cake (with Dryad).

..

7/50

Piece of cake (with Dryad).

..

7/50

Piece of cake (with Dryads).

..

8/50

Piece of cake (with Dryads).

..

8/50

Piece of cake (with Dryads).

..

8/50

Previous efforts.

LAVA.

• Linaro Automated Validation Architecture• Automation system for deploying operating systems• Virtual and physical hardware supported• Allows running boot, bootloader and system level tests

9/50

Kernel CI.

https://kernelci.org/st6ts/

10/50

LAVA LMP.

https://linux.codehelp.co.uk/the-problem-of-sd-mux.html11/50

SD MUX.

12/50

SD MUX – open hardware.

https://git.tizen.org/cgit/tools/testl6b/sd-mux 13/50

Autohat board (SD MUX-based).

https://github.com/resin-io/6utoh6t-bo6rd14/50

SD MUX issues.

$ dmesg | t6il -[ . ] usb - : new full-speed USB device number using xhci_hcd[ . ] usb - : device descriptor re6d/ , error -[ . ] usb - : device descriptor re6d/ , error -[ . ] usb - : new full-speed USB device number using xhci_hcd[ . ] usb - : Device not responding to setup 6ddress.[ . ] usb - : Device not responding to setup 6ddress.[ . ] usb - : device not 6ccepting 6ddress , error -[ . ] usb - : new full-speed USB device number using xhci_hcd[ . ] usb - : Device not responding to setup 6ddress.[ . ] usb - : Device not responding to setup 6ddress.[ . ] usb - : device not 6ccepting 6ddress , error -[ . ] usb usb -port : un6ble to enumer6te USB device

15/50

Idea.

Constraints.

• Only replaceable media

• No single point of failure parts

• No USB involvement(from test server)

16/50

Requirements.

• Minimum external connections

• Unified remote accessto target devices

• Easy setup and maintenance

17/50

Features.

• User interface (often requested)

• Power measurement (increasing demand)

• Writing EDID to HDMI

18/50

Hardware.

MuxPi.

19/50

MuxPi components.

..

20/50

MuxPi components.

..

20/50

MuxPi components.

..

20/50

MuxPi components.

..

20/50

MuxPi components.

..

20/50

MuxPi components.

..

20/50

MuxPi components.

..

20/50

MuxPi components.

..

20/50

MuxPi components.

..

20/50

MuxPi components.

..

20/50

MuxPi components.

..

20/50

Essential MuxPi functions.

• Switching a microSD card between DUT and TS

• Switching power supply for DUT

• Switching jumpers/buttons of DUT

• Measuring power consumption of DUT

• Writing EDID to DUT over HDMI connection

• Providing DUT connection (UART, USB, ETH, microSD card) over Ethernet

• Interacting with farm maintainer

21/50

Indicators.

22/50

Easy maintenance.

23/50

Extensibility.

24/50

Major improvements.

• Independent (standalone)

• Aware of its state

• Easy to maintain

• Extensible from start

25/50

Building your own.

NanoPi NEO ≈ $10

PartsSoldering skills HighPatience A LOT

https://git.tizen.org/cgit/tools/muxpi

26/50

Building your own.

NanoPi NEO ≈ $10

Parts ≈ $80

Soldering skills HighPatience A LOT

https://git.tizen.org/cgit/tools/muxpi

26/50

Building your own.

NanoPi NEO ≈ $10

Parts ≈ $80

Soldering skills High

Patience A LOT

https://git.tizen.org/cgit/tools/muxpi

26/50

Building your own.

NanoPi NEO ≈ $10

Parts ≈ $80

Soldering skills HighPatience A LOT

https://git.tizen.org/cgit/tools/muxpi

26/50

Building your own.

NanoPi NEO ≈ $10

Parts ≈ $80

Soldering skills HighPatience A LOT

https://git.tizen.org/cgit/tools/muxpi

26/50

Software.

Multitier architecture.

• “Do One Thing and Do It Well”

• RESTful HTTP APIs

• Homogeneous solution stack

27/50

Responsibilities.

• Who knows what requires verification?

Perun

• Who knows which actions are necessary?

Weles

• Who knows where can it be done?

Boruta

• Who knows how to do it?

Dryad (MuxPi-based)

28/50

Responsibilities.

• Who knows what requires verification?

Perun

• Who knows which actions are necessary?

Weles

• Who knows where can it be done?

Boruta

• Who knows how to do it?

Dryad (MuxPi-based)

28/50

Responsibilities.

• Who knows what requires verification?

Perun

• Who knows which actions are necessary?

Weles

• Who knows where can it be done?

Boruta

• Who knows how to do it?

Dryad (MuxPi-based)

28/50

Responsibilities.

• Who knows what requires verification?

Perun

• Who knows which actions are necessary?

Weles

• Who knows where can it be done?Boruta

• Who knows how to do it?

Dryad (MuxPi-based)

28/50

Responsibilities.

• Who knows what requires verification?

Perun

• Who knows which actions are necessary?

Weles

• Who knows where can it be done?Boruta

• Who knows how to do it?Dryad (MuxPi-based)

28/50

Dryad (farm).

• Manages single DUT

• Fully aware of its capabilities

• Requires only two interfaces• Power supply• Network connection (Ethernet)

....

29/50

Dryad (software).

$ fot6 --helpUs6ge of fot6:

-c6rd stringp6th to SDc6rd

-m6p stringp6th to JSON form6tted m6pping

-md stringURL to MD SUMS file

-quietsuppress logging

$ stm --helpUs6ge of stm:

-dutconnect SD c6rd to DUT

-m dur6tiontime del6y for tick comm6nd

-tickpower off 6nd on 6fter 'm' (s)

-tsconnect SD c6rd to test server

....

30/50

Dryad (software).

$ fot6 --helpUs6ge of fot6:

-c6rd stringp6th to SDc6rd

-m6p stringp6th to JSON form6tted m6pping

-md stringURL to MD SUMS file

-quietsuppress logging

$ stm --helpUs6ge of stm:

-dutconnect SD c6rd to DUT

-m dur6tiontime del6y for tick comm6nd

-tickpower off 6nd on 6fter 'm' (s)

-tsconnect SD c6rd to test server

.....

30/50

Dryad (software).

$ fot6 --helpUs6ge of fot6:

-c6rd stringp6th to SDc6rd

-m6p stringp6th to JSON form6tted m6pping

-md stringURL to MD SUMS file

-quietsuppress logging

$ stm --helpUs6ge of stm:

-dutconnect SD c6rd to DUT

-m dur6tiontime del6y for tick comm6nd

-tickpower off 6nd on 6fter 'm' (s)

-tsconnect SD c6rd to test server

....

30/50

Dryad (software).

$ fot6 --helpUs6ge of fot6:

-c6rd stringp6th to SDc6rd

-m6p stringp6th to JSON form6tted m6pping

-md stringURL to MD SUMS file

-quietsuppress logging

$ stm --helpUs6ge of stm:

-dutconnect SD c6rd to DUT

-m dur6tiontime del6y for tick comm6nd

-tickpower off 6nd on 6fter 'm' (s)

-tsconnect SD c6rd to test server

....

30/50

Boruta.

• Dryad farm management system

• Schedules requests• Priority• Device groups• Delayed access

• Provides convenient accessto selected Dryad

31/50

Dryad life cycle in Boruta.

.

.

.

Actionsperformed

.

.

Unallocated

.

.

Matchesrequirements

.

.

Preparesenvironment

. .Sets uptunnel

.

.

.

Maintenancemode

.32/50

Boruta on stack.

........

33/50

Boruta on stack.

........

33/50

Boruta on stack.

........

33/50

Boruta on stack.

........

33/50

Boruta on stack.

........

33/50

Boruta on stack.

........

33/50

Boruta on stack.

........

33/50

Weles.

• Lightweight testing framework

• Provides LAVA-like interface

• YAML job definition 7→

actions executed on DUT• Deploy• Boot• Test• Collect

34/50

Weles purpose.

.

.

.

Actionsperformed

.

.

Unallocated

.

.

Matchesrequirements

.

.

Preparesenvironment

. .Sets uptunnel

.

.

.

Maintenancemode

.35/50

Weles action sequence.

.

.

.

ParsesYAML

.

.

Collectsassets

.

.

RequestsDUT

.

.

Performstests

36/50

Weles on stack.

...........

37/50

Weles on stack.

...........

37/50

Weles on stack.

...........

37/50

Weles on stack.

...........

37/50

Weles on stack.

...........

37/50

Weles on stack.

...........

37/50

Perun.

• OS images testing system

• Schedules verification(per new set of OS images)

• Automates QA step ofRelease Engineering Duty

38/50

Perun action sequence.

.

.

.

CrawlURL

.

.

Reportchanges

.

.

SubmitWelesjobs

.

.

Collectartifacts

. .Interpretresults

39/50

Perun on stack.

...........

40/50

Perun on stack.

...........

40/50

SLAV.

...........

41/50

Keeping SLAV simple.

42/50

Keeping SLAV simple (and decoupled).

43/50

Keeping SLAV simple (and decoupled).

43/50

Keeping SLAV simple (and decoupled).

43/50

Keeping SLAV simple (and decoupled).

43/50

Next steps.

Future plans.

Hardware• Audio I/O

• USB Type C investigation

• NanoPi serial consoleon USB

Software• Web interfacesfor current layers

• Service state management

• Release engineer's layer

44/50

Further details.

• MuxPihttps://wiki.tizen.org/MuxPi

• SD MUX (deprecated – lesson learnt)https://wiki.tizen.org/SD_MUX

45/50

Help?.

• Mailing listgeneral@lists.tizen.org

• #tizen on Freenodehttps://webchat.freenode.net/?channels=tizen

46/50

Conclusion.

Summary.

• Quick setup

• Easy maintenance

• Responsibilities division

• Execution parallelization

• Environment unification

47/50

Questions?

47/50

Thank you!

Paweł Wieczorekp.wieczorek2@samsung.com

Samsung R&D Institute Poland

Acknowledgements

• Metropolis – simple, modern Beamer theme

Pictures used

• https://en.wikipedi6.org/wiki/File:Heckert_GNU_white.svg• https://commons.wikimedi6.org/wiki/File:Tux.svg• https://commons.wikimedi6.org/wiki/File:W6yl6nd_Logo.svg• https://commons.wikimedi6.org/wiki/File:Enlightenment_logo_bl6ck.png• https://developer.tizen.org/sites/def6ult/files/im6ges/6bout_tizen_ .png• https://pix6b6y.com/en/security-industri6l-logistic- /• https://commons.wikimedi6.org/wiki/File:ColoredBl6nkM6p-World- E.svg• https://commons.wikimedi6.org/wiki/File:Nuvol6_Kore6n_fl6g.svg• https://commons.wikimedi6.org/wiki/File:Nuvol6_Polish_fl6g.svg• https://commons.wikimedi6.org/wiki/File:Nuvol6_USA_fl6g.svg• https://v6lid6tion.lin6ro.org/st6tic/docs/v /_im6ges/l6v6.svg• https://wiki.lin6ro.org/Pl6tform/LAB/LMP_in_pr6ctice• https://forums.resin.io/uplo6ds/resin/origin6l/ X/ 6b e cd b b f6 ede ce b 6df6 .jpg• https://commons.wikimedi6.org/wiki/File:It6li6n_tr6ffic_signs_-_ferm6rsi_e_d6re_precedenz6_-_stop.svg• https://f6rm .st6ticflickr.com/ / _d b 6e _k_d.jpg• https://pix6b6y.com/en/st6rs-new-6dvertisement-sign- /• https://pix6b6y.com/en/upd6te-upgr6de-renew-improve- /• https://www.goodfreephotos.com/6lbums/vector-im6ges/different-colored-l6yers-vector-file.png• https://pix6b6y.com/en/ethernet-internet-l6n-network- /• https://pix6b6y.com/en/power-c6ble-plug-socket- /• https://pix6b6y.com/en/memory-c6rd-sd-secure-digit6l- /• https://pix6b6y.com/en/kiss-lips-mouth-red-love-ros6- /• https://commons.wikimedi6.org/wiki/File:PEO-smiley_smile.svg