Introduction

OBS can build not only build traditional RPM packages, but it can even build appliances or installation ISO images.

We use this feature in YaSTadd-emojiinstaller project to build an installation image with the latest YaST (which is not included in Factory yet). This allows as to try new things without worrying to break the installer in Factory or in the upcoming openSUSE distribution.

The Problem

Because the public build service is sometimes quite slow I have deployed a private OBS instance (see the OBS download page) at my workstation.

I found out that product building is relatively slow, even when running in the private instance locally. A quick look revealed that for building ISO image the build service downloads too many packages including source packages, debugsource and debuginfo packages which are not need for our installation ISO image. The amount of the unnecessary packages is several gigabytes even for our small mini installation DVD (which contains packages only for Minimal X pattern, no KDE nor GNOME...).

Focus of the Project

This project is focused on decreasing package download for product ISO image build.

The project should make building ISO images in OBS faster (especially in a private instance linking to the public OBS server), saving disk space and download bandwidth.

Result

I have successfully added kiwi file parsing (to know whether the debuginfo and source packages are needed or not) and skipping debuginfo and source packages into both OBS scheduler (which checks and evaluates dependencies and triggers builds) and worker (which downloads the packages during package/ISO build).

Outcome of the Project

  • Significantly decreased amount of downloaded packages, over 7GB (!!) even for our small miniDVD
  • This results in faster builds, disk space and network bandwidth savings
  • Created a GitHub pull request to include this improvement upstream
  • I learned a lot about OBS internals
  • And refreshed my perl knowledge ;-)

Looking for mad skills in:

obs kiwi build iso_image

This project is part of:

Hack Week 10


Comments

  • lslezak
    about 6 years ago by lslezak | Reply

    I have implemented this optimization in my private OBS instance at https://muffin.suse.cz, for building our miniDVD.

    In this case it saves over 7GB package download for the ISO image build!!

  • lslezak
    about 6 years ago by lslezak | Reply

    I have created a GitHub pull request to include this improvement in the next OBS release (and likely in the public instance as well), see https://github.com/openSUSE/open-build-service/pull/426.

  • lslezak
    about 6 years ago by lslezak | Reply

    Details for the miniDVD build:

    Skipped 3831 (3299 MB) debuginfo packages
    Skipped 1511 (3793 MB) source packages
    

    That's pretty nice saving!

  • lslezak
    about 6 years ago by lslezak | Reply

    There are still possible some more improvements, OBS still downloads some more not needed packages:

    • If you build x86_64 only medium also i586 packages will be downloaded although not needed
    • It downloads packages from all repositories, if the kiwi file contains more repositories the packages will be downloaded from all of them, but during build kiwi will properly choose only one package (depending on the repository priority attribute)
    • It downloads all binary packages which are build from a source package. For example if your kiwi file only requires e.g. augeas package in the target product ISO then OBS will still download augeas-devel, augeas-devel-32bit, augeas-lenses and augeas-lenses-tests packages.

Similar Projects

obsci - a CI for open build service packages by tbechtold

Currently it's pretty difficult to create tests...


Migrate more OBS service scripts to pure systemd by enavarro_suse

Following the work started in the last hackweek...


Ruby Gem: Open Build Service API by mschnitzer

I just recently started a new project which is ...


Ant Ivy OBS resolver by cbosdonnat

SUSE Manager uses an ivy repository on the R&D ...


MicroOS Desktop by RBrownSUSE

[Video Recording of openSUSE Conference sessio...


MicroOS Desktop by RBrownSUSE

[Video Recording of openSUSE Conference sessio...