This is my second post of my Google Summer of Code 2018 series. Links for the previous posts can be found below:
On the last post, I reported an issue with the OBS API package where the assets pipeline would trigger an error during compilation. Digging further, I realized that the problem lies within the fact that the assets group used in the package Gemfile was removed from Rails 4.
Removing the group from the Gemfile does solve the problem:
sed -i 's/^group :assets do$//' /usr/share/obs/api/Gemfile sed -i '93s/^end$//' /usr/share/obs/api/Gemfile
But then, Debian testing has a different version of the Ruby sass gem, which leads to a different error when compiling the assets for the API. A dirty fix would be to update the version in the Gemfile.lock before compiling the assets:
sed -i 's/sass (3.4.22)/sass (3.4.23)/' /usr/share/obs/api/Gemfile.lock
After a few days working on this issue, I realized there is a script in the OBS API package, provided by the OBS Debian packagers, that:
/usr/share/obs/api/script/rake-tasks.sh script does most of the work I
have prepared on my local salt scripts to properly set the API, with one
drawback: is it idempotent? i.e.: can we safely run the script every time we
provision our nodes keeping the same state on every run? Running the create and
setup rake tasks several times will trigger several warnings. I had solved the
issue (before knowing about the rake-tasks.sh script) by making sure that the
setup task would run only once by using salt grains. The problem of using the
same approach for the rake-tasks.sh script is that if the obs-api package gets
updated, the ownership of the package files will be set back to root and apache
won’t be able to serve them, hence, we would need to re-run the related tasks
available in the rake-tasks.sh script.
A possible solution for the issue would be to keep the rake-tasks.sh script running only once (on the first setup) and replicate the permissions setup present in the file for whenever the OBS package is upgraded.
My OBS salt configuration repository for local OBS deploy is available here.
Following up on the last post topic on how to fetch information about new uploads to the Debian archive, and if I understood the debian package workflow correctly, dak is the software the FTP masters use to accept packages in the archive. Checking dak’s code, I realized dak fires an email to recipients upon package acceptance. One of the recipients of these emails is the debian-devel-changes mailing list. Hence, it led me to believe that the shortest path to watching new uploads in the archive is to simply watch the mailing list. If we consider that email protocols do not guarantee a message was delivered, we would be accepting some margin of false negatives, where a package was accepted in the archive and an email was not fired (of course there would also be the cases where the mailing service fails).
The Debian IRC bot, which announces new archive uploads, just subscribes to the aforementioned mailing list and announces the upload whenever a new email is received.
When the time comes, we will also watch this mailing list to trigger new Clang builds in our OBS instance.
I could finally set up a local default Debian packages builder with a fully functional OBS instance (server, worker, api) by setting the configurations steps provided by Héctor Orón Martínez
When using OBS Download on Demand feature, my workers are still getting stuck at some point (no logs at all; to be debugged).
Next, I will proceed to aid Reshabh and Sylvestre on configuring our OBS instance at opencollab/llvm-slave-salt And start customizing a OBS project to build packages with Clang.