The title of this project is inspired from the must-see video

John Hughes - Testing the Hard Stuff and Staying Sane

I would like to play with QuickCheck to generate tests and possibly find bugs automatically.

I chose Wicked as the tested software, but it could be anything else.

My hope is to stay away from Haskell as much as possible as I have no real interest in learning Yet Another Programming Language. Ideally, I would reuse as much as possible from the existing Ruby testsuite for Wicked and stick to Ruby. There are projects like RubyCheck that are ports of QuickCheck to Ruby. But well, if I have to learn Haskell as part of this project, I would do it.

Feel welcome to join the project.

Looking for hackers with the skills:

wicked testing quickcheck

This project is part of:

Hack Week 13


Comments

  • mvidner
    over 4 years ago by mvidner | Reply

    Regarding QuickCheck and Ruby, Balazs wrote something last year but I don't know how far he got: https://github.com/bkutil/exploring_quickcheck

  • e_bischoff
    over 4 years ago by e_bischoff | Reply

    Yes, apart from the choice of examples (Wicked in my case), it is more or less the same project.

    And as a matter of fact, I don't want to start with Wicked straight ahead. The plan was to implement in a first step more or less the same example as in the video : it would be an external C program that manages a queue (of integers, why not). I hope to do as many bugs as they do in the video :-) so RubyCheck has at least something to find.

    I would use Rantly too to generate the testcase integers. So yeah, a lot of similarities. Thanks for pointing that.

  • e_bischoff
    over 4 years ago by e_bischoff | Reply

    I just realized that Rantly implements QuickCheck too, it is not only for generating random data.

    So there are three out there: Rantly, RushCheck, RubyCheck . Not sure yet which one to take, that will be part of the exploration.

  • e_bischoff
    over 4 years ago by e_bischoff | Reply

    Nope, rantly does not implement QuickCheck, despite what it says. It is just a very convenient random input generator.

    Getting back to RubyCheck as initially planned.

  • e_bischoff
    over 4 years ago by e_bischoff | Reply

    RubyCheck too does no test failure reduction. Rantly at least has "shrinking", but it is shrinking of data types (strings, integers...), not a sequence of test cases... Do I have to map manually a sequence of test cases to such a data type?

    I also discovered Theft. Continuing exploration...

  • e_bischoff
    over 4 years ago by e_bischoff | Reply

    I ussceeded reducing (shrinking) with Rantly. Starting to save my experiments in my gitlab home project.

  • e_bischoff
    over 4 years ago by e_bischoff | Reply

    Hackweek finished. I did not make it in time testing Wicked with that way.

    I wrote a report with all my finding in the gitlab project:

    https://gitlab.suse.de/ebischoff/generate_tests/blob/master/hackweek.txt

  • e_bischoff
    over 4 years ago by e_bischoff | Reply

    I would need 3 days to finish the hackweek project and do its application to wicked.

  • e_bischoff
    about 4 years ago by e_bischoff | Reply

    I'll consider the project as finished - Rantly has been improved, with my patches accepted upstream, and I played with the tool enough to know quite well his strengths and weaknesses.

    If I want to test wicked with it, it could be a separate hackweek project.

Similar Projects

labgrid: add support for sispmctl and remote ykush access by mbrugger

labgrid [0] is an embedded board control python...


Testing GNU/Linux distributions on Uyuni by juliogonzalezgil

Join the rocket chat channel! [https://chat.su...


libuitest - a generic GUI testing library by dancermak

Testing GUIs is hard: unit tests require a ...


ethtool ops for netdevsim by mkubecek

This can be seen as a subproject of [ethtool ne...