Automated tests with bbi_daily

BBI stands for BrainVISA Build Infrastructure. The casa_distro_admin bbi_daily command orchestrates this process in a given BBE (BrainVISA Build Environment, a directory that serves as a base directory for casa-distro). The bbi_daily command will run the following main steps, while (optionally) logging detailed output to a Jenkins server:

  1. Self-update casa_distro with git pull;
  2. Update the casa-run and casa-dev images from the BrainVISA website;
  3. Run a compilation in a dev environment (bv_maker sources, bv_maker configure, bv_maker build, bv_maker doc);
  4. Run all tests that would be run by bv_maker test in the dev environment;
  5. Install the compiled software in a new user image based on the casa-run image (the software is installed under /casa/install);
  6. Run the tests in the user image.

Walkthrough

Here is a detailed log of how nightly builds are set up in NeuroSpin. You can take inspiration from it to create your own personalized set-up.

  1. Install singularity. Configure the fakeroot functionality of Singularity, as explained in the Singularity Admin Guide. In short, you need admin rights on your machine, and you have to run:

    sudo singularity config fakeroot --add $USER

    (with $USER being a-sac-ns-brainvisa in this case).

  2. Create a directory dedicated to the nightly builds and change to it:

    export CASA_BASE_DIRECTORY=/volatile/a-sac-ns-brainvisa/bbi_nightly
mkdir "$CASA_BASE_DIRECTORY"
cd "$CASA_BASE_DIRECTORY"

    This directory will be the base directory for your BrainVISA Build Environment (BBE).

  3. Create the jenkins_auth file in the BBE base directory. This text file must consist of two lines: the first line contains the username on the Jenkins server, the second line contains the token that can be generated in the Jenkins web interface (click on the user on top right corner, select Configure and create a new API token). This token will be used instead of the account password to upload build results. One advantage of using a token is the possibility to revoke it at any time. It is recommended to use one token per BrainVISA Build Environment and name the token accordingly.

    warning:

    You should make this file non-readable by others, e.g. create it with:

    chmod 0600 jenkins_auth

  4. Download the casa-dev image using the link found on the Downloads page of <https://brainvisa.info/>. Also download the associated JSON file (to be extra safe, check that the md5sum of the image matches that stored in the JSON file). Do the same for the casa-run image (it may not appear on the Downloads page, just replace -dev- by -run- in the URL of the dev image):

    wget https://brainvisa.info/download/casa-dev-5.0.sif
wget https://brainvisa.info/download/casa-dev-5.0.sif.json
md5sum casa-dev-5.0.sif
wget https://brainvisa.info/download/casa-run-5.0.sif
wget https://brainvisa.info/download/casa-run-5.0.sif.json
md5sum casa-run-5.0.sif

  5. Create a directory for your development environment and set it up:

    cd "$CASA_BASE_DIRECTORY"
mkdir brainvisa-master-5.0
singularity run --bind ./brainvisa-master-5.0:/casa/setup \
    casa-dev-5.0.sif distro=brainvisa branch=master

  6. Edit the conf/svn.secret file with your BioProj login and password.

  7. Check out and compile an initial build:

    "$CASA_BASE_DIRECTORY"/brainvisa-master-5.0/bin/bv_maker

  8. Create an inital user image. You may need to set the SINGULARITY_TMPDIR environment variable to a disk with enough free space (about twice the size of the final user image):

    export SINGULARITY_TMPDIR=/volatile/tmp
"$CASA_BASE_DIRECTORY"/brainvisa-master-5.0/bin/casa_distro_admin \
    create_user_image \
    version=nightly \
    environment_name=brainvisa-master-5.0 \
    name=brainvisa-master-5.0-userimage
    • environment_name is the name of the development environment.
    • name is the full name of the created user image. We change it from the default, because we need it to be fully explicit: this name will be the name of the build on the Jenkins page.

  9. Install the environment for your new user image:

    cd "$CASA_BASE_DIRECTORY"
mkdir brainvisa-master-5.0-userimage
singularity run --bind ./brainvisa-master-5.0-userimage:/casa/setup \
    brainvisa-master-5.0-userimage.sif

  10. Add the branch and image_version keys to the conf/casa_distro.json of the user environment: these variables are not set by setup_user, but they are necessary for bbi_daily to make the correct link between the environment of the user image and the corresponding dev environment.

    note:Issue #246 tracks progress on that issue, this workaround will become obsolete once that issue is fixed.

  11. Put the reference test data in place. Best is to copy it from a known-good source. Beware that it must be copied in both environments:

    • "$CASA_BASE_DIRECTORY"/brainvisa-master-5.0/tests/ref/
    • "$CASA_BASE_DIRECTORY"/brainvisa-master-5.0-userimage/tests/ref/

  12. Check that the whole bbi_daily process is able to run successfully:

    "$CASA_BASE_DIRECTORY"/brainvisa-master-5.0/bin/casa_distro_admin \
    bbi_daily

    Beware that the output of each step is displayed only when that step is finished, so the command may seem to hang for a long time.

  13. Set the bbi_daily command to run on a regular basis using crontab -e:

    MAILTO=your.email@host.example
37 5 * * * PATH=/usr/local/bin:/usr/bin:/bin CASA_BASE_DIRECTORY=/volatile/a-sac-ns-brainvisa/bbi_nightly SINGULARITY_TMPDIR=/volatile/tmp /volatile/a-sac-ns-brainvisa/bbi_nightly/brainvisa-master-5.0/bin/casa_distro_admin bbi_daily jenkins_server='https://brainvisa.info/builds'
    note:Remember to set all the needed environment variables, including BRAINVISA_PUBLISH_SERVER if needed. PATH may need to be set additionally, in case your Singularity installation is under /usr/local (by default cron limits PATH to /usr/bin:/bin).