The production instance of our Open Build Service can be found here
This is the seventh post of my Google Summer of Code 2018 series. Links for the previous posts can be found below:
My GSoC contributions can be seen at the following links
As I have been reporting on previous posts, whenever one tries to build packages against Debian unstable with Debian Stretch OBS packages, the OBS scheduler would download the package dependencies and clean up the cacher before the downloads were completed. This would cause builds to never get triggered: OBS would keep downloading dependencies forever.
This would only happen for builds on debian sid (unstable) and buster (testing).
After some investigation (as already reported before), we realized that the OBS version packaged in Debian 9 (the one we are currently using) does not support debian source packages built with dpkg >= 1.19.
While the backports package included the patch needed to support source packages built with newer versions of dpkg, we would still get the same issue with unstable and testing builds.
We spent a lot of time stuck in this issue, and I ended up proposing to host the whole archive in our OBS instance so we could use it as dependencies for new packages. It ended up being a terrible idea, since it would be unfeasible to test it locally and we would have to update the packages on each accepted upload into debian unstable (note that several packages get updated every day). Hence, I kept studying OBS code to understand how builds get triggered and why it would not happen for unstable/testing packages.
After diving into OBS code, we realized that it uses libsolv to read repositories. In libsolv’s change history, we found that there was an arbitrary size limit to the debian control file, which was fixed in an newer version of the package.
After updating the libsolv0, libsolv-perl and libsolvext0 packages (using
Debian Buster versions), we could finally trigger Debian unstable builds.
With OBS being able to trigger builds for Debian Unstable, we now needed to be able to substitute builds with gcc for builds with the clang compiler. As suggested by my mentor, Sylvestre, we wanted to substitute gcc binaries for clang ones. For that, we needed to find a way to run a script suggest by Sylvestre in our build environments prior triggering the builds.
Fortunately, Open Build Service has a Source Services
feature
that can do exactly that: change source code before builds or run specific
tasks in the build environment during build time. For the latter (which is what
we needed to substitute gcc binaries), OBS requires one to build an
obs-service-SERVICE_NAME package, which will be injected in the build
environment as a dependency and get whatever task specified by that package
performed.
We built our obs-service-clang-build package with the needed scripts to perform the gcc/clang substitutions in build time.
To activate the substitution task for a package, it should contain a _service
file with in its root path (along with the debian and pristine sources) the
following content:
<services>
<service name="clang_build" mode="buildtime" />
</services>
If everything is correct, we should see
[ TIME in seconds] Running build time source services...
in the log file, followed by the gcc substitution commands.
Now, we must proceed to trigger Debian unstable builds for newly accepted uploads. To do so, we wrote a cron job to monitor the debian-devel-changes mailing list. We check for new source uploads to debian unstable and trigger a new build for those new packages (we added a 4 hour cool-down before triggering the build to allow the package to propagate through the mirrors).
The results can be seen at https://irill8.siege.inria.fr/project/show/Debian:Unstable:Clang
Debian distro-tracker has links to clang builds, which would point to the former service at clang.debian.net. We now want to substitute those links to point to our new OBS instance. Thus, we opened a new pull request to perform such change.
Now that we have an instance of our project up and running, there are a few tasks left for our final GSoC submission.
