Piece of cake

testing remote embedded devicesmade easy with MuxPi

.

Paweł Wieczorek

February 3, 2018

Samsung R&D Institute Poland

Agenda.

1. Introduction

2. Previous efforts

3. Idea

4. Hardware

5. Software

6. Next steps

7. Conclusion1/49

Introduction.

Tizen.

2/49

Use cases.

https://news.s6msung.com/glob6l/tizen- - -first-milestone-rele6se-to-open-new-opportunities-in-the-iot-er6

3/49

Release engineering.

• Continuousplatform development

• QA step prior pullingnew changes

• Package internal testsare not enough

4/49

Remote accessibility.

.....

• Easy to store in a secure manner

• Less effort than per developer

• Better utilized when shared

5/49

Piece of cake (with MuxPi).

6/49

Piece of cake (with Dryad).

..

7/49

Piece of cake (with Dryad).

..

7/49

Piece of cake (with Dryads).

..

8/49

Piece of cake (with Dryads).

..

8/49

Piece of cake (with Dryads).

..

8/49

Previous efforts.

LAVA.

• Linaro Automated Validation Architecture

• Automation system for deploying operating systems

• Virtual and physical hardware supported

• Allows running boot, bootloader and system level tests

9/49

Kernel CI.

https://kernelci.org/st6ts/

10/49

LAVA LMP.

https://linux.codehelp.co.uk/the-problem-of-sd-mux.html11/49

SD MUX.

12/49

SD MUX – open hardware.

https://git.tizen.org/cgit/tools/testl6b/sd-mux

13/49

Autohat board (SD MUX-based).

https://github.com/resin-io/6utoh6t-bo6rd

14/49

SD MUX issues.

$ dmesg | t6il -[ . ] usb - : new full-speed USB device number using xhci_hcd[ . ] usb - : device descriptor re6d/ , error -[ . ] usb - : device descriptor re6d/ , error -[ . ] usb - : new full-speed USB device number using xhci_hcd[ . ] usb - : Device not responding to setup 6ddress.[ . ] usb - : Device not responding to setup 6ddress.[ . ] usb - : device not 6ccepting 6ddress , error -[ . ] usb - : new full-speed USB device number using xhci_hcd[ . ] usb - : Device not responding to setup 6ddress.[ . ] usb - : Device not responding to setup 6ddress.[ . ] usb - : device not 6ccepting 6ddress , error -[ . ] usb usb -port : un6ble to enumer6te USB device

15/49

Idea.

Constraints.

• Only replaceable media

• No single point of failure parts

• No USB involvement(from test server)

16/49

Requirements.

• Minimum external connections

• Unified remote accessto target devices

• Easy setup and maintenance

17/49

Features.

• User interface(often requested)

• Power measurement(increasing demand)

• Writing EDID to HDMI

18/49

Hardware.

MuxPi.

19/49

MuxPi components.

..

20/49

MuxPi components.

..

20/49

MuxPi components.

..

20/49

MuxPi components.

..

20/49

MuxPi components.

..

20/49

Essential MuxPi functions.

• Switching a microSD card between DUT and TS

• Switching power supply for DUT

• Switching jumpers/buttons of DUT

• Measuring power consumption of DUT

• Writing EDID to DUT over HDMI connection

• Providing DUT connection (UART, USB, ETH,microSD card) over Ethernet

• Interacting with farm maintainer

21/49

Indicators.

22/49

Easy maintenance.

23/49

Extensibility.

24/49

Major improvements.

• Independent(standalone)

• Aware of its state

• Easy to maintain

• Extensible from start

25/49

Building your own.

NanoPi NEO ≈ $10

PartsSoldering skills HighPatience A LOT

https://git.tizen.org/cgit/tools/muxpi

26/49

Building your own.

NanoPi NEO ≈ $10

Parts ≈ $80

Soldering skills HighPatience A LOT

https://git.tizen.org/cgit/tools/muxpi

26/49

Building your own.

NanoPi NEO ≈ $10

Parts ≈ $80

Soldering skills High

Patience A LOT

https://git.tizen.org/cgit/tools/muxpi

26/49

Building your own.

NanoPi NEO ≈ $10

Parts ≈ $80

Soldering skills HighPatience A LOT

https://git.tizen.org/cgit/tools/muxpi

26/49

Building your own.

NanoPi NEO ≈ $10

Parts ≈ $80

Soldering skills HighPatience A LOT

https://git.tizen.org/cgit/tools/muxpi

26/49

Software.

Multitier architecture.

• “Do One Thing and Do It Well”

• RESTful HTTP APIs

• Homogeneous solution stack

27/49

Responsibilities.

• Who knows what requires verification?

Perun

• Who knows which actions are necessary?

Weles

• Who knows where can it be done?

Boruta

• Who knows how to do it?

MuxPi

28/49

Responsibilities.

• Who knows what requires verification?

Perun

• Who knows which actions are necessary?

Weles

• Who knows where can it be done?

Boruta

• Who knows how to do it?

MuxPi

28/49

Responsibilities.

• Who knows what requires verification?

Perun

• Who knows which actions are necessary?

Weles

• Who knows where can it be done?

Boruta

• Who knows how to do it?

MuxPi

28/49

Responsibilities.

• Who knows what requires verification?

Perun

• Who knows which actions are necessary?

Weles

• Who knows where can it be done?Boruta

• Who knows how to do it?

MuxPi

28/49

Responsibilities.

• Who knows what requires verification?

Perun

• Who knows which actions are necessary?

Weles

• Who knows where can it be done?Boruta

• Who knows how to do it?MuxPi

28/49

MuxPi (farm).

• Manages single DUT

• Fully aware of its capabilities

• Requires only two interfaces• Power supply• Network connection (Ethernet)

....

29/49

MuxPi (software).

$ fot6 --helpUs6ge of fot6:-c6rd string

p6th to SDc6rd-m6p string

p6th to JSON form6tted m6pping-md string

URL to MD SUMS file-quiet

suppress logging

$ stm --helpUs6ge of stm:

-dutconnect SD c6rd to DUT

-m dur6tiontime del6y for tick comm6nd

-tickpower off 6nd on 6fter 'm' (s)

-tsconnect SD c6rd to test server

....

30/49

MuxPi (software).

$ fot6 --helpUs6ge of fot6:-c6rd string

p6th to SDc6rd-m6p string

p6th to JSON form6tted m6pping-md string

URL to MD SUMS file-quiet

suppress logging

$ stm --helpUs6ge of stm:

-dutconnect SD c6rd to DUT

-m dur6tiontime del6y for tick comm6nd

-tickpower off 6nd on 6fter 'm' (s)

-tsconnect SD c6rd to test server

.....

30/49

MuxPi (software).

$ fot6 --helpUs6ge of fot6:-c6rd string

p6th to SDc6rd-m6p string

p6th to JSON form6tted m6pping-md string

URL to MD SUMS file-quiet

suppress logging

$ stm --helpUs6ge of stm:

-dutconnect SD c6rd to DUT

-m dur6tiontime del6y for tick comm6nd

-tickpower off 6nd on 6fter 'm' (s)

-tsconnect SD c6rd to test server

....

30/49

MuxPi (software).

$ fot6 --helpUs6ge of fot6:-c6rd string

p6th to SDc6rd-m6p string

p6th to JSON form6tted m6pping-md string

URL to MD SUMS file-quiet

suppress logging

$ stm --helpUs6ge of stm:

-dutconnect SD c6rd to DUT

-m dur6tiontime del6y for tick comm6nd

-tickpower off 6nd on 6fter 'm' (s)

-tsconnect SD c6rd to test server

....

30/49

Boruta.

• Dryad farm management system

• Schedules requests• Priority• Device groups• Delayed access

• Provides convenient accessto selected Dryad

31/49

Boruta on stack.

.....

32/49

Dryad life cycle in Boruta.

.

.

.

Actionsperformed

.

.

Unallocated

.

.

Matchesrequirements

.

.

Preparesenvironment

. .Sets uptunnel

.

.

.

Maintenancemode

.

33/49

Weles.

• Lightweight testing framework

• Provides LAVA-like interface

• YAML job definition 7→

actions executed on DUT• Deploy• Boot• Test• Collect

34/49

Weles on stack.

........

35/49

Weles purpose.

.

.

.

Actionsperformed

.

.

Unallocated

.

.

Matchesrequirements

.

.

Preparesenvironment

. .Sets uptunnel

.

.

.

Maintenancemode

.

36/49

Weles action sequence. .

.

.

ParseYAML

.

.

Collectassets

.

.

RequestDUT

.

.

Performtests

37/49

Perun.

• OS images testing system

• Schedules verification(per new set of OS images)

• Automates QA step ofRelease Engineering Duty

38/49

Perun on stack.

...........

39/49

Perun action sequence.

.

.

.

CrawlURL

.

.

Reportchanges

.

.

SubmitWelesjobs

.

.

Collectartifacts

. .Interpretresults

40/49

Keeping it simple.

41/49

Keeping it simple (and decoupled).

42/49

Keeping it simple (and decoupled).

42/49

Keeping it simple (and decoupled).

42/49

Keeping it simple (and decoupled).

42/49

Next steps.

Future plans.

Hardware• Audio I/O

• USB Type Cinvestigation

• NanoPi serial consoleon USB

Software• Web interfacesfor current layers

• Service statemanagement

• Release engineer's layer

43/49

Further details.

• MuxPihttps://wiki.tizen.org/MuxPi

• SD MUX (deprecated – lesson learnt)https://wiki.tizen.org/SD_MUX

44/49

Help?.

• Mailing listgeneral@lists.tizen.org

• #tizen on Freenodehttps://webchat.freenode.net/?channels=tizen

45/49

Conclusion.

Summary.

• Quick setup

• Easy maintenance

• Responsibilities division

• Execution parallelization

• Environment unification

46/49

Questions?

46/49

Thank you!

Paweł Wieczorekp.wieczorek2@samsung.com

Samsung R&D Institute Poland

Acknowledgements

• Metropolis – simple, modern Beamer theme

Pictures used

• https://en.wikipedi6.org/wiki/File:Heckert_GNU_white.svg• https://commons.wikimedi6.org/wiki/File:Tux.svg• https://commons.wikimedi6.org/wiki/File:W6yl6nd_Logo.svg• https://commons.wikimedi6.org/wiki/File:Enlightenment_logo_bl6ck.png• https://developer.tizen.org/sites/def6ult/files/im6ges/6bout_tizen_ .png• https://pix6b6y.com/en/security-industri6l-logistic- /• https://commons.wikimedi6.org/wiki/File:ColoredBl6nkM6p-World- E.svg• https://commons.wikimedi6.org/wiki/File:Nuvol6_Kore6n_fl6g.svg• https://commons.wikimedi6.org/wiki/File:Nuvol6_Polish_fl6g.svg• https://commons.wikimedi6.org/wiki/File:Nuvol6_USA_fl6g.svg• https://v6lid6tion.lin6ro.org/st6tic/docs/v /_im6ges/l6v6.svg• https://wiki.lin6ro.org/Pl6tform/LAB/LMP_in_pr6ctice• https://forums.resin.io/uplo6ds/resin/origin6l/ X/

6b e cd b b f6 ede ce b 6df6 .jpg• https://commons.wikimedi6.org/wiki/File:

It6li6n_tr6ffic_signs_-_ferm6rsi_e_d6re_precedenz6_-_stop.svg• https://f6rm .st6ticflickr.com/ / _d b 6e _k_d.jpg• https://pix6b6y.com/en/st6rs-new-6dvertisement-sign- /• https://pix6b6y.com/en/upd6te-upgr6de-renew-improve- /• https://www.goodfreephotos.com/6lbums/vector-im6ges/

different-colored-l6yers-vector-file.png• https://pix6b6y.com/en/ethernet-internet-l6n-network- /• https://pix6b6y.com/en/power-c6ble-plug-socket- /• https://pix6b6y.com/en/memory-c6rd-sd-secure-digit6l- /• https://pix6b6y.com/en/kiss-lips-mouth-red-love-ros6- /• https://commons.wikimedi6.org/wiki/File:PEO-smiley_smile.svg