paul eggleton intel open source technology centre · keeping it green: integrated qa with the yocto...
TRANSCRIPT
![Page 1: Paul Eggleton Intel Open Source Technology Centre · Keeping it Green: Integrated QA with the Yocto Project Paul Eggleton Intel Open Source Technology Centre ELCE 2013 Edinburgh 24](https://reader033.vdocuments.mx/reader033/viewer/2022042910/5f3ea2dff7cf076fa8105751/html5/thumbnails/1.jpg)
Keeping it Green: Integrated QA with the Yocto Project
Paul EggletonIntel Open Source Technology Centre
ELCE 2013 ● Edinburgh ● 24 Oct 2013
![Page 2: Paul Eggleton Intel Open Source Technology Centre · Keeping it Green: Integrated QA with the Yocto Project Paul Eggleton Intel Open Source Technology Centre ELCE 2013 Edinburgh 24](https://reader033.vdocuments.mx/reader033/viewer/2022042910/5f3ea2dff7cf076fa8105751/html5/thumbnails/2.jpg)
Yocto Project | The Linux Foundation2/27
![Page 3: Paul Eggleton Intel Open Source Technology Centre · Keeping it Green: Integrated QA with the Yocto Project Paul Eggleton Intel Open Source Technology Centre ELCE 2013 Edinburgh 24](https://reader033.vdocuments.mx/reader033/viewer/2022042910/5f3ea2dff7cf076fa8105751/html5/thumbnails/3.jpg)
Yocto Project | The Linux Foundation3/27
![Page 4: Paul Eggleton Intel Open Source Technology Centre · Keeping it Green: Integrated QA with the Yocto Project Paul Eggleton Intel Open Source Technology Centre ELCE 2013 Edinburgh 24](https://reader033.vdocuments.mx/reader033/viewer/2022042910/5f3ea2dff7cf076fa8105751/html5/thumbnails/4.jpg)
Yocto Project | The Linux Foundation4/27
![Page 5: Paul Eggleton Intel Open Source Technology Centre · Keeping it Green: Integrated QA with the Yocto Project Paul Eggleton Intel Open Source Technology Centre ELCE 2013 Edinburgh 24](https://reader033.vdocuments.mx/reader033/viewer/2022042910/5f3ea2dff7cf076fa8105751/html5/thumbnails/5.jpg)
Yocto Project | The Linux Foundation5/27
?
![Page 6: Paul Eggleton Intel Open Source Technology Centre · Keeping it Green: Integrated QA with the Yocto Project Paul Eggleton Intel Open Source Technology Centre ELCE 2013 Edinburgh 24](https://reader033.vdocuments.mx/reader033/viewer/2022042910/5f3ea2dff7cf076fa8105751/html5/thumbnails/6.jpg)
Yocto Project | The Linux Foundation6/27
![Page 7: Paul Eggleton Intel Open Source Technology Centre · Keeping it Green: Integrated QA with the Yocto Project Paul Eggleton Intel Open Source Technology Centre ELCE 2013 Edinburgh 24](https://reader033.vdocuments.mx/reader033/viewer/2022042910/5f3ea2dff7cf076fa8105751/html5/thumbnails/7.jpg)
Yocto Project | The Linux Foundation7/27
Yocto Project
● Extend and improve the core of OpenEmbedded (OE-Core)● Develop additional tools around it● Releases on a 6 month cadence● Help make building an embedded Linux OS easier
![Page 8: Paul Eggleton Intel Open Source Technology Centre · Keeping it Green: Integrated QA with the Yocto Project Paul Eggleton Intel Open Source Technology Centre ELCE 2013 Edinburgh 24](https://reader033.vdocuments.mx/reader033/viewer/2022042910/5f3ea2dff7cf076fa8105751/html5/thumbnails/8.jpg)
Yocto Project | The Linux Foundation8/27
Why automated testing?
● Confidence that changes haven't introduced regressions● Humans don't need to do what a machine could be doing● Avoid everyone re-inventing the wheel● Developers can run tests on their own machines
![Page 9: Paul Eggleton Intel Open Source Technology Centre · Keeping it Green: Integrated QA with the Yocto Project Paul Eggleton Intel Open Source Technology Centre ELCE 2013 Edinburgh 24](https://reader033.vdocuments.mx/reader033/viewer/2022042910/5f3ea2dff7cf076fa8105751/html5/thumbnails/9.jpg)
Yocto Project | The Linux Foundation9/27
Runtime testing (1.5+)
● New framework for 1.5• Replaces old shell-based “imagetest” framework• Python-based (unittest)• Tests run on the build host, run commands on target over SSH• QEMU targets only (for now)
![Page 10: Paul Eggleton Intel Open Source Technology Centre · Keeping it Green: Integrated QA with the Yocto Project Paul Eggleton Intel Open Source Technology Centre ELCE 2013 Edinburgh 24](https://reader033.vdocuments.mx/reader033/viewer/2022042910/5f3ea2dff7cf076fa8105751/html5/thumbnails/10.jpg)
Yocto Project | The Linux Foundation10/27
Runtime testing (1.5+)
● Easy to run• Automatically after building an image• Explicitly
● Choose which tests to run● Minimal intrusion into CI server setup
• (and doesn't require one)
![Page 11: Paul Eggleton Intel Open Source Technology Centre · Keeping it Green: Integrated QA with the Yocto Project Paul Eggleton Intel Open Source Technology Centre ELCE 2013 Edinburgh 24](https://reader033.vdocuments.mx/reader033/viewer/2022042910/5f3ea2dff7cf076fa8105751/html5/thumbnails/11.jpg)
Yocto Project | The Linux Foundation11/27
Runtime testing (1.5+)
● Easy to extend• Written in Python• Unimportant details abstracted away• Extensible with standard layer mechanism
![Page 12: Paul Eggleton Intel Open Source Technology Centre · Keeping it Green: Integrated QA with the Yocto Project Paul Eggleton Intel Open Source Technology Centre ELCE 2013 Edinburgh 24](https://reader033.vdocuments.mx/reader033/viewer/2022042910/5f3ea2dff7cf076fa8105751/html5/thumbnails/12.jpg)
Yocto Project | The Linux Foundation12/27
Running tests
● To enable running tests automatically after building any image:
• In local.conf:TEST_IMAGE = “1”
● To run manually:• In local.conf:
INHERIT += “testimage”• Each time:$ bitbake -c testimage <imagename>
![Page 13: Paul Eggleton Intel Open Source Technology Centre · Keeping it Green: Integrated QA with the Yocto Project Paul Eggleton Intel Open Source Technology Centre ELCE 2013 Edinburgh 24](https://reader033.vdocuments.mx/reader033/viewer/2022042910/5f3ea2dff7cf076fa8105751/html5/thumbnails/13.jpg)
Yocto Project | The Linux Foundation13/27
Finding tests
● Core tests located in meta/lib/oeqa/runtime● Additional layers can add their own
• Add python modules in lib/oeqa/runtime under the layer• Should not use same names as modules in OE-Core
● Specify tests to run by setting TEST_SUITES variable
![Page 14: Paul Eggleton Intel Open Source Technology Centre · Keeping it Green: Integrated QA with the Yocto Project Paul Eggleton Intel Open Source Technology Centre ELCE 2013 Edinburgh 24](https://reader033.vdocuments.mx/reader033/viewer/2022042910/5f3ea2dff7cf076fa8105751/html5/thumbnails/14.jpg)
Yocto Project | The Linux Foundation
Example test cases
meta/lib/oeqa/dmesg.py
meta/lib/oeqa/df.py
meta/lib/oeqa/smart.py
![Page 15: Paul Eggleton Intel Open Source Technology Centre · Keeping it Green: Integrated QA with the Yocto Project Paul Eggleton Intel Open Source Technology Centre ELCE 2013 Edinburgh 24](https://reader033.vdocuments.mx/reader033/viewer/2022042910/5f3ea2dff7cf076fa8105751/html5/thumbnails/15.jpg)
Yocto Project | The Linux Foundation
Other automated testing features
![Page 16: Paul Eggleton Intel Open Source Technology Centre · Keeping it Green: Integrated QA with the Yocto Project Paul Eggleton Intel Open Source Technology Centre ELCE 2013 Edinburgh 24](https://reader033.vdocuments.mx/reader033/viewer/2022042910/5f3ea2dff7cf076fa8105751/html5/thumbnails/16.jpg)
Yocto Project | The Linux Foundation16/27
ptest
● Run tests from upstream software package● Provides packaging and installation of test suites● Runner script for the target (run-ptest)● Idea is to coalesce test output to the same format
![Page 17: Paul Eggleton Intel Open Source Technology Centre · Keeping it Green: Integrated QA with the Yocto Project Paul Eggleton Intel Open Source Technology Centre ELCE 2013 Edinburgh 24](https://reader033.vdocuments.mx/reader033/viewer/2022042910/5f3ea2dff7cf076fa8105751/html5/thumbnails/17.jpg)
Yocto Project | The Linux Foundation17/27
Autobuilder
● Continuous Integration server● Based on Buildbot● Configured to run builds & tests out of the box
![Page 18: Paul Eggleton Intel Open Source Technology Centre · Keeping it Green: Integrated QA with the Yocto Project Paul Eggleton Intel Open Source Technology Centre ELCE 2013 Edinburgh 24](https://reader033.vdocuments.mx/reader033/viewer/2022042910/5f3ea2dff7cf076fa8105751/html5/thumbnails/18.jpg)
Yocto Project | The Linux Foundation18/27
bitbake-selftest
● Simple test script for bitbake itself● Helps avoid regressions when working on bitbake● Quick to run
![Page 19: Paul Eggleton Intel Open Source Technology Centre · Keeping it Green: Integrated QA with the Yocto Project Paul Eggleton Intel Open Source Technology Centre ELCE 2013 Edinburgh 24](https://reader033.vdocuments.mx/reader033/viewer/2022042910/5f3ea2dff7cf076fa8105751/html5/thumbnails/19.jpg)
Yocto Project | The Linux Foundation19/27
Other scripts
● test-reexec• Tests re-execution of tasks
● test-dependencies.sh• Finds “floating” auto-detected dependencies
![Page 20: Paul Eggleton Intel Open Source Technology Centre · Keeping it Green: Integrated QA with the Yocto Project Paul Eggleton Intel Open Source Technology Centre ELCE 2013 Edinburgh 24](https://reader033.vdocuments.mx/reader033/viewer/2022042910/5f3ea2dff7cf076fa8105751/html5/thumbnails/20.jpg)
Yocto Project | The Linux Foundation
Future plans
![Page 21: Paul Eggleton Intel Open Source Technology Centre · Keeping it Green: Integrated QA with the Yocto Project Paul Eggleton Intel Open Source Technology Centre ELCE 2013 Edinburgh 24](https://reader033.vdocuments.mx/reader033/viewer/2022042910/5f3ea2dff7cf076fa8105751/html5/thumbnails/21.jpg)
Yocto Project | The Linux Foundation21/27
Future (1.6+) - Runtime tests
● Runtime tests on real hardware• Simple – one or two machines, directly attached• Advanced – number of machines attached to test server (“test
rack”)• “push model” vs “pull model”
● ptest integration• Report collection / analysis
● Additional runtime tests● GUI tests
![Page 22: Paul Eggleton Intel Open Source Technology Centre · Keeping it Green: Integrated QA with the Yocto Project Paul Eggleton Intel Open Source Technology Centre ELCE 2013 Edinburgh 24](https://reader033.vdocuments.mx/reader033/viewer/2022042910/5f3ea2dff7cf076fa8105751/html5/thumbnails/22.jpg)
Yocto Project | The Linux Foundation22/27
Future (1.6+)
● Non-runtime tests• “oe-selftest”• Converting current manual test cases into automated tests
![Page 23: Paul Eggleton Intel Open Source Technology Centre · Keeping it Green: Integrated QA with the Yocto Project Paul Eggleton Intel Open Source Technology Centre ELCE 2013 Edinburgh 24](https://reader033.vdocuments.mx/reader033/viewer/2022042910/5f3ea2dff7cf076fa8105751/html5/thumbnails/23.jpg)
Yocto Project | The Linux Foundation23/27
Social effects
● Improve communication between development & QA● Opportunities for QA to become more directly involved
![Page 24: Paul Eggleton Intel Open Source Technology Centre · Keeping it Green: Integrated QA with the Yocto Project Paul Eggleton Intel Open Source Technology Centre ELCE 2013 Edinburgh 24](https://reader033.vdocuments.mx/reader033/viewer/2022042910/5f3ea2dff7cf076fa8105751/html5/thumbnails/24.jpg)
Yocto Project | The Linux Foundation24/27
Summary
● New testing framework● More testing-related improvements to come● Focus on quality● Send us feedback● Get involved!
![Page 25: Paul Eggleton Intel Open Source Technology Centre · Keeping it Green: Integrated QA with the Yocto Project Paul Eggleton Intel Open Source Technology Centre ELCE 2013 Edinburgh 24](https://reader033.vdocuments.mx/reader033/viewer/2022042910/5f3ea2dff7cf076fa8105751/html5/thumbnails/25.jpg)
Yocto Project | The Linux Foundation
Questions?
![Page 26: Paul Eggleton Intel Open Source Technology Centre · Keeping it Green: Integrated QA with the Yocto Project Paul Eggleton Intel Open Source Technology Centre ELCE 2013 Edinburgh 24](https://reader033.vdocuments.mx/reader033/viewer/2022042910/5f3ea2dff7cf076fa8105751/html5/thumbnails/26.jpg)
Thank you for your participation!
www.yoctoproject.org
![Page 27: Paul Eggleton Intel Open Source Technology Centre · Keeping it Green: Integrated QA with the Yocto Project Paul Eggleton Intel Open Source Technology Centre ELCE 2013 Edinburgh 24](https://reader033.vdocuments.mx/reader033/viewer/2022042910/5f3ea2dff7cf076fa8105751/html5/thumbnails/27.jpg)
Image credits:
"Plateau Forest (2)" by Nicholas_Thttp://www.flickr.com/photos/nicholas_t/9249259415/
"Stop Fracking!" by greensefahttp://www.flickr.com/photos/greensefa/8205936358/
"Money" by aresauburnhttp://www.flickr.com/photos/aresauburnphotos/2678453389