Currently the upstream LTP is executed by a hacked up and old runltp script that executes even worse and fairly old mess called ltp-pan which in turn actually executes the test cases. This whole thing is a unmaintainable mess that should have been replaced with something simpler a long time ago. It should also have a few more features that has been requested in the meantime and not implemented since nobody wants to touch the code. For instance executing the test cases on a different machine via ssh and writing the results locally. Another feature I've been thinking about for quite some time is a parallel test execution, since most of the test cases in fact could be executed in parallel which could easily speed up the test run twice. There are other tests that cannot, mostly stress tests, but also test cases that modify global system state, i.e. system time, make use of sysv IPC, use loop devices, etc. These kind of tests should be annotated somehow so that we do not end up with a test cases competing for a global resources in a parallel test run.

Looking for mad skills in:

hw15challenge

This project is part of:

Hack Week 15 Hack Week 17


Comments

  • metan
    almost 2 years ago by metan | Reply

    The proof of concept code is at:

    https://github.com/metan-ucw/ltp

    After installation you can try it by running the runltp-ng script, either with -o option that runs open posix testsuite or with a path to a syscall runtest file as a last argument.

    What I managed to finish:

    • Working proof of concent LTP testrunner that more or less can do everything that the old one could have done and more:
      • testcases are killed on timeout and the test run carries on!
      • the test runner can produce nice clickable (javascript) html page with results
      • the test runtime is measured and included in results

    What is still on TODO:

    • Parallel test execution (but the infrastructure is there!)
    • Remove test execution (should be easy, since we have clean enough codebase now)

  • metan
    almost 2 years ago by metan | Reply

    Example test runner output:

    (it's still missing information about hardware/OS release/etc. which will be done later)

    http://metan.ucw.cz/outgoing/ltp-openposix.html

    http://metan.ucw.cz/outgoing/ltp-syscalls.html

  • metan
    8 months ago by metan | Reply

    The requirements have changed a bit meanwhile, so basically it's needed to split the testrunner and tested systems into two different machines to avoid incomplete results on kernel crashes.

    I will try to prototype proof of concept that will take a path to a qcow image of an virtual machine, execute testcases there and produce results.

  • metan
    7 months ago by metan | Reply

    The prototype has been finished, it's a bit messy perl code but the basic functionality is there.

    Here you can see output put from the testruns with a kernel trace in the middle followed by a reboot of the VM:

    http://metan.ucw.cz/outgoing/log.html

    The work is for now in my home repo in GitHub at:

    https://github.com/metan-ucw/ltp/tree/master/tools/runltp-ng

Similar Projects

Exploring Workflows in JIRA by rsblendido

Lay out a mock version of an existing SUSE R&...