The kernel compile and link itself is only a fraction of the OBS kernel-default package build. Several other passes take significant time: - Post build checks, especially the clamav scan - Debuginfo extractioon, which is single-threaded - Writing the RPMs

The post build checks and clamav are disabled in the KOTD projects, but they are still annoying when doing official builds. Clamav is suspiciously slow and should be investigated. The RPM write speed has been improved by switching to bzip2 compression, at the expense of their increased size. There is still room for improvement by writing the RPMs in parallel. Debuginfo extraction is a major pain.

Looking for mad skills in:

Nothing? Add some keywords!

This project is part of:

Hack Week 14

Activity

  • over 2 years ago: alnovak liked Faster kernel builds in the OBS
  • over 2 years ago: mkubecek liked Faster kernel builds in the OBS
  • over 2 years ago: wanghaisu liked Faster kernel builds in the OBS
  • over 2 years ago: ZRen liked Faster kernel builds in the OBS
  • over 2 years ago: a_faerber liked Faster kernel builds in the OBS
  • Show History

    Comments

    • michal-m
      over 2 years ago by michal-m | Reply

      I started looking into the clamav scan, because it seemed like low-hanging fruit. I have a patch to the clamscan utility to scan the files in parallel, I just need to polish it and submit upstream. I also fixed some issues in the post-build-checks-malwarescan package. I'll test this and submit. Let's see if I can do something about the debuginfo extraction. I tried a few years ago had to give up. But I was young back then :-P.

    • michal-m
      over 2 years ago by michal-m | Reply

      clamav-devel pull request to add parallel mode to clamscan.

    • michal-m
      over 2 years ago by michal-m | Reply

      I also submitted the post-build-checks-malwarescan improvements to Base:System (SR#405470).

    • michal-m
      over 2 years ago by michal-m | Reply

      In homeadd-emojibranchesadd-emojiSLE-12:Update in the IBS, during kernel-default build, the clamav scan takes 99 + 154 seconds. That's a considerable improvement, since it normally takes something like 6 + 7 minutes. There is still the issue that we run all the post-build checks twice if module signing is enabled. These numbers are with the patched clamav, which is not yet submitted.

    • michal-m
      over 2 years ago by michal-m | Reply

      I managed to parallelize find-debuginfo.sh as well, hooray. I have yet to submit the patch upstream, but I think I finished the goals I set for this hackweek. So I'll mark it as done.

    • michal-m
      over 2 years ago by michal-m | Reply

      Upstream submission of the find-debuginfo.sh patches. I also uploaded the patched rpm to homeadd-emojibranchesadd-emojiSLE-12:Update in the IBS, let's see how fast is the kernel going to build.

    • michal-m
      over 2 years ago by michal-m | Reply

      Bummer. I found a race in the find-debuginfo.sh patch. Testing a new version.

    Similar Projects

    This project is one of its kind!