Some notes on the OBS Documentation

This is my fourth post of my Google Summer of Code 2018 series. Links for the previous posts can be found below:

• Post 1: My Google Summer of Code 2018 project
• Post 2: Setting up a local OBS development environment
• Post 3: Running OBS Workers and OBS staging instance

Open Build Service Manuals

OBS provides several manuals on their web site, including an admin and a user guide. Since I needed to travel to an academic conference last week (too many hours in airplanes), I took some time to read the full OBS documentation to have a better understanding of the tool we have been deploying. While reading the documentation, I took some notes on relevant points for our GSoC project (and sent a patch to fix a few typos in OBS documentaion, which I discuss below.

Hardware requirements

There is no need to distribute all different services in OBS server since our instance will not process heavy build loads. We do want to separate the server services from the OBS workers (package builders) so expensive builds will not compromise our server performance.

According to OBS documentation, we need

• 1 core for each scheduler architecture
• 4GB ram for each scheduler architecture
• 50GB disk per architecture for each build distribution supported

We are working with a single build distribution (Debian unstable). Therefore, we need 50GB disk for our OBS instance for each supported architecture (unless we want to mirror the whole distribution instead of using the Download on Demand OBS feature).

We would like to work with 3 different architectures: i686, x86_64 and arm. Hence, we need 150GB, 12GB ram and 3 cores according to the OBS admin guide.

Summary:

• 12GB RAM
• 150GB disk
• 3 cores

OBS Instance Configuration

We want to change some instance configurations like

• Change OBS instance description
• Set administrator email
• Disable new users sign up: since all builds in this OBS instance will be fired automatically and no new projects will be configured for now, we will not allow people to create accounts in our OBS instance.

It is important to note that the proper way of changing a project’s configuration is through the API calls. Therefore, we will need to make such calls in our salt scripts.

To list OBS configurations:

osc -A https://irill8.siege.inria.fr api /configuration

To redefine OBS configurations:

osc -A https://irill8.siege.inria.fr api /configuration -T new_config_file.xml

Workers configuration

OBS workers need to be allowed to connect to the server in /etc/obs/BSConfig.pm. The server accepts connections from any node in the network by default, but we can (and should) force OBS to accept connections only from our own nodes.

Source Services

OBS provide a way to run scripts to change sources before builds. This may be useful for building against Clang.

To create a source service, we must create a script in the /usr/lib/obs/service/ directory and create a new _service file either in the package or in the project repository level.

_service is a XML file pointing to our script under /usr/lib/obs/service/ and providing possible parameters to the script:

<services>
 <service name="foobar.sh" mode="MODE">
 <param name="PARAMETER1">PARAMETER1_VALUE</param>
 </service>
</services>

Self signed certificates

For testing purposes, there is no need to generate proper SSL certificates, we can generate and self sign our own:

mkdir /srv/obs/certs
openssl genrsa -out /srv/obs/certs/server.key 1024
openssl req -new -key /srv/obs/certs/server.key -out /srv/obs/certs/server.csr
openssl x509 -req -days 365 -in /srv/obs/certs/server.csr -signkey /srv/obs/certs/server.key -out /srv/obs/certs/server.crt
cat /srv/obs/certs/server.key /srv/obs/certs/server.crt > /srv/obs/certs/server.pem

Finally, we must trust our certificate:

cp /srv/obs/certs/server.pem /etc/ssl/certs/
c_rehash /etc/ssl/certs/

Message bus

OBS supports rabbitMQ usage to publish events such as build results, package updates, etc. In the future, we could also set a rabbitMQ instance so other services can listen to a queue with our Clang build results.

Next steps (A TODO list to keep on the radar)

• Write patches for the OBS worker issue described in post 3
• Configure Debian projects on OBS with salt, not manually
• Change the default builder to perform builds with clang
• Trigger new builds by using the dak/mailing lists messages
• Verify the rake-tasks.sh script idempotency and propose patch to opencollab repository
• Separate salt recipes for workers and server (locally)
• Properly set hostnames (locally)