The title of this project is inspired from the must-see video

John Hughes - Testing the Hard Stuff and Staying Sane

I would like to play with QuickCheck to generate tests and possibly find bugs automatically.

I chose Wicked as the tested software, but it could be anything else.

My hope is to stay away from Haskell as much as possible as I have no real interest in learning Yet Another Programming Language. Ideally, I would reuse as much as possible from the existing Ruby testsuite for Wicked and stick to Ruby. There are projects like RubyCheck that are ports of QuickCheck to Ruby. But well, if I have to learn Haskell as part of this project, I would do it.

Feel welcome to join the project.

Looking for mad skills in:

wicked testing quickcheck

This project is part of:

Hack Week 13

Activity

  • over 2 years ago: thardeck disliked Don't write tests! Generate them.
  • over 3 years ago: wawiv74 joined Don't write tests! Generate them.
  • over 3 years ago: thardeck liked Don't write tests! Generate them.
  • over 3 years ago: dgutu liked Don't write tests! Generate them.
  • over 3 years ago: mvidner liked Don't write tests! Generate them.
  • Show History

    Comments

    • mvidner
      over 3 years ago by mvidner | Reply

      Regarding QuickCheck and Ruby, Balazs wrote something last year but I don't know how far he got: https://github.com/bkutil/exploring_quickcheck

    • e_bischoff
      over 3 years ago by e_bischoff | Reply

      Yes, apart from the choice of examples (Wicked in my case), it is more or less the same project.

      And as a matter of fact, I don't want to start with Wicked straight ahead. The plan was to implement in a first step more or less the same example as in the video : it would be an external C program that manages a queue (of integers, why not). I hope to do as many bugs as they do in the video :-) so RubyCheck has at least something to find.

      I would use Rantly too to generate the testcase integers. So yeah, a lot of similarities. Thanks for pointing that.

    • e_bischoff
      over 3 years ago by e_bischoff | Reply

      I just realized that Rantly implements QuickCheck too, it is not only for generating random data.

      So there are three out there: Rantly, RushCheck, RubyCheck . Not sure yet which one to take, that will be part of the exploration.

    • e_bischoff
      over 3 years ago by e_bischoff | Reply

      Nope, rantly does not implement QuickCheck, despite what it says. It is just a very convenient random input generator.

      Getting back to RubyCheck as initially planned.

    • e_bischoff
      over 3 years ago by e_bischoff | Reply

      RubyCheck too does no test failure reduction. Rantly at least has "shrinking", but it is shrinking of data types (strings, integers...), not a sequence of test cases... Do I have to map manually a sequence of test cases to such a data type?

      I also discovered Theft. Continuing exploration...

    • e_bischoff
      over 3 years ago by e_bischoff | Reply

      I ussceeded reducing (shrinking) with Rantly. Starting to save my experiments in my gitlab home project.

    • e_bischoff
      over 3 years ago by e_bischoff | Reply

      Hackweek finished. I did not make it in time testing Wicked with that way.

      I wrote a report with all my finding in the gitlab project:

      https://gitlab.suse.de/ebischoff/generate_tests/blob/master/hackweek.txt

    • e_bischoff
      over 3 years ago by e_bischoff | Reply

      I would need 3 days to finish the hackweek project and do its application to wicked.

    • e_bischoff
      about 3 years ago by e_bischoff | Reply

      I'll consider the project as finished - Rantly has been improved, with my patches accepted upstream, and I played with the tool enough to know quite well his strengths and weaknesses.

      If I want to test wicked with it, it could be a separate hackweek project.

    Similar Projects

    Controlling and Testing the YaST UI Remotely (for Integration Tests, openQA) by lslezak

    Hackweek 18 Update

    What Has Been Done ...


    ethtool ops for netdevsim by mkubecek

    This can be seen as a subproject of [ethtool ne...


    Make "salt-toaster" available to be used outside SUSE by PSuarezHernandez

    The salt-toaster (https://github.com/openSUSE...


    Dudenetes by pgeorgiadis

    ![alt text](https://dudeism.com/wp-content/uplo...