openQA has a well earned reputation as a 'full system' testing tool, able to test a system end-to-end from the operating system to it's applications on a number of different platforms and architectures, including VM's & Bare Metal.
But one area of weakness is it's usefulness as a testing tool for developers or packagers. openQA can easily test a package once it's INSIDE a distribution, but how do you test that package BEFORE submitting it to the distribution?
openSUSE/SLE has the concept of 'staging', where we utilise OBS and openQA to basically create 'what-if' builds's for the whole OS where we can do basic validation tests to ensure a proposed check-in doesn't break the OS or it's basic functionality, but staging doesn't test the package itself
This hackweek project will experiment with solutions to this problem.
Concepts to be investigated include
- Providing reference disk images for package-specific tests for each OS (SLE, Tumbleweed, Leap)
- Injecting test code into openQA without needing it to be submitted via github. This might include openQA tests being stored as part of the OBS package.
- Loading package-specific tests in openQA by feeding openQA variables over its REST API
- Booting these reference images to either a console or X11 as required by the test
- Testing and providing the results in a meaningful way to the Developer
I will try to find solutions to the above which will work in the following contexts
- a Developer working 'packageless' with just a git repo and a local compiler (unlikely, but still want to think about it)
- a Developer working on a package in their OBS Home/Devel project
- Release Management/Devel Project reviewers of an OBS submit request wanting to see the results of Package Testing BEFORE deciding whether or not the package can be checked-in or forwarded to staging
Looking for mad skills in:
perl openqa obs
This project is part of:
Hack Week 15