Google Summer of Code

This year I was accepted in the Google Summer of Code project as a student under the Debian project umbrella. I will keep a journal of my progress in this blog and hope to publish a link with the production instance of our project by the end of the summer.

The project

You can see the full project proposal here.

Debian package builds with clang are performed from time to time through massive rebuilds of the Debian archive on AWS. The results of these builds are published on clang.debian.net. My summer project aims to automate Debian archive clang rebuilds by substituting the current clang build service with the Open Build System (OBS). By the end of the summer, the clang links on the right corner of QA package pages will point to a new OBS clang CI, which will constantly provide up to date clang build results of the whole Debian archive.

Mentor

My mentor is Sylvestre Ledru, he is based in France and does work with Firefox and the LLVM projects other than Debian.

About OBS

The Open Build Service is a system designed to build and distribute binary packages from sources. It supports building different package formats, including RPM and Debian packages. OBS is maintained by the openSUSE project and has been widely used by different companies such as Dell and Intel. OBS backend is written in perl and its API uses Ruby on Rails. You can find OBS code repositories on github.com/openSUSE/open-build-service

Related project on LLVM

Reshabh Sharma is a student working on GSoC under the LLVM umbrella. His project aims to migrate apt.llvm.org, a Jenkins instance used to build and distribute LLVM Debian packages, to OBS. Sylvestre Ledru is also mentoring Reshabh on this project, therefore we decided to use the same OBS instance for both projects.

First issues to solve

During the first week of GSoC, I have been working on two issues:

Debian OBS package

The OBS packages available in Debian testing do install cleanly, with one issue: the assets pipeliline compilation step fails, meaning there is a problem when providing static content like css and javascript files for the OBS front-end Rails application.

My first idea was to use the packages available in debian unstable, which has a newer version of OBS. Unfortunatelly, these pakages are broken at the moment: The new OBS version uses Rails 5, which is not available yet.

We can now either work on the assets compilation issue on Debian testing or work on the Debian unstable packages. In this comming week, I will interact with the OBS packages maintainer (as suggested by Sylvestre) to better understand the packages and get/offer help with them.

The salt scripts I have been using to deploy OBS in my development environment are available at my github account.

Fetching new uploads information

Sylvestre pointed out that one of the key issues for this project is to detect new uploads in the Debian archive. They were using a RSS feed on gmane generated from a mailing list. We want to find out what the best way of fetching such packages would be. After contacting Lucas Kanashiro (Debian Developer) about the issue, he suggested I should check the Debian Continuous Integration project code (debci) since it does need to fetch packages from the archive. I will have better answers on this matter on my next post, although I believe I will need to check a couple more Debian applications to come up with the best approach to detect new uploads.

Next steps

I want to come up with a clean, replicable OBS deploy with salt using the packages available in Debian (either testing or unstable). With that, we can deploy OBS to a testing instance where other people can audit our work or join us setting up the new service. For that, I will contact the OBS maintainer as already pointed out. I will also take a couple hours on the upcoming week to check more Debian applications code to come up with the best approach to fetch new uploads information.