Installation and setup¶

Setting up Docker¶

Install and use docker on an Ubuntu host¶

Install docker using apt-get :
```
sudo apt-get install docker.io
```
To enable users other than root and users with sudo access to be able to run Docker commands:

Warning

Users who can run Docker commands have effective root control of the system. Only grant this privilege to trusted users.

The following procedure applies to version 1.5 and later of Docker.
1. Create the docker group (maybe group docker already exists):
sudo groupadd docker
1. Restart the docker service:
sudo service docker restart
Warning

For some os system version (Ubuntu 14.04-15.10) use docker.io instead of docker

Warning

The UNIX socket /var/run/docker.sock is now readable and writable by members of the docker group.
1. Add the users that should have Docker access to the docker group:
sudo usermod -a -G docker <username>
1. logout / login to update groups cache
Or use the following command to open a new shell forcing to take group updates into account:
sudo su <login>

Change Docker’s storage base directory¶

By default, docker images are stored in will be /var/lib/docker/aufs but can fall back to btrfs, devicemapper or vfs.

/var/lib/docker/{driver-name} will contain the driver specific storage for contents of the images.
/var/lib/docker/graph/ now only contains metadata about the image, in the json and layersize files.

In the case of aufs:

/var/lib/docker/aufs/diff/ has the file contents of the images.
/var/lib/docker/repositories-aufs is a JSON file containing local image information. This can be viewed with the command docker images.

First method

You can change Docker’s storage base directory (where container and images go) using the -g option when starting the Docker daemon. You must to stop docker:
sudo service docker stop
Create a new directory for docker:
sudo mkdir /mnt/docker

Ubuntu/Debian: edit your /etc/default/docker file with the -g option:
DOCKER_OPTS="-dns 8.8.8.8 -dns 8.8.4.4 -g /mnt/docker" # (or write it if the line doesn't exist in this file)
Fedora/Centos: edit /etc/sysconfig/docker, and add the -g option in the other_args variable: ex.
other_args="-g /var/lib/ testdir".
If there’s more than one option, make sure you enclose them in " ".
Docker should use the new directory after a restart:
sudo service docker start
You can check it using:
docker info

Second method (Using a symlink)

Warning

These steps depend on your current /var/lib/docker being an actual directory (not a symlink to another location).

Stop docker:
service docker stop.
Verify no docker process is running:
ps faux
Double check docker really isn’t running. Take a look at the current docker directory:
ls /var/lib/docker/
Make a backup:
tar -zcC /var/lib docker > /mnt/pd0/var_lib_docker-backup-$(date +%s).tar.gz
Move the /var/lib/docker directory to your new partition:
mv /var/lib/docker /mnt/pd0/docker
Make a symlink:
ln -s /mnt/pd0/docker /var/lib/docker
Take a peek at the directory structure to make sure it looks like it did before the mv:
ls /var/lib/docker/
(note the trailing slash to resolve the symlink)
Start docker back up service
docker start
restart your containers

Overview of the existing public brainvisa images¶

To search available images on docker hub (example with ubuntu) :

docker search --stars=10 ubuntu

or using this url: https://hub.docker.com

An open source brainvisa repository is available on docker hub: https://hub.docker.com/r/cati

Note

It is a public repository !

cati/casa-test image

Minimal OS system to test a package of brainvisa in lambda-user conditions.

Just some libraries are installed to run a X server and to test the creation of snapshots.

Several images:
1. Ubuntu 12.04
2. Ubuntu 16.04
3. windows-7-32: Ubuntu 14.04 + Wine for windows-7-32
4. windows-7-64: Ubuntu 14.04 + Wine for windows-7-64
cati/casa-dev image

Based on cati/casa-test image.

Include all system dependencies (using apt get) to run a compilation of brainvisa and Qt Installer Framework to create a brainvisa package.

These images are dedicated for developers.

Several images:
1. Ubuntu 12.04
2. Ubuntu 16.04
3. windows-7-32: Ubuntu 14.04 + Wine for windows-7-32
4. windows-7-64: Ubuntu 14.04 + Wine for windows-7-64

How to use a docker image¶

Get docker image:

docker pull cati/casa-test:ubuntu-12.04

Examples with other cati images in docker hub :

docker pull cati/casa-test:ubuntu-16.04
docker pull cati/casa-dev:ubuntu-12.04
docker pull cati/casa-dev:ubuntu-16.04

Run a docker image:

docker run -it --rm cati/casa-dev:ubuntu-16.04-bug_fix /bin/bash

Cleaning up docker¶

Containers¶

Remove exited containers

docker ps --filter status=dead --filter status=exited -aq | xargs -r docker rm -v

Remove older containers (example: 2 weeks or more)

docker ps --filter "status=exited" | grep 'weeks ago' | awk '{print $1}' | xargs --no-run-if-empty sudo docker rm

Remove all containers
```
docker rm $(docker ps -a -q)
```

Images¶

Remove an image:

$ docker images
REPOSITORY                  TAG                    IMAGE ID            CREATED             VIRTUAL SIZE
cati/casa-dev       ubuntu-12.04           7c1691e1e9d1        2 days ago          2.264 GB

To know the id of the image to remove…

docker rmi 7c1691e1e9d1

To remove cati/casa-dev.

If one or more containers are using the image, use the option -f to force the command rmi:

docker rmi -f 7c1691e1e9d1

Remove unused images

docker images --no-trunc | grep '<none>' | awk '{ print $3 }' | xargs -r docker rmi

Remove all images
```
docker rmi $(docker images -q)
```

How to change the development environment ?¶

To add an external library, modify the Dockerfile of casa-dev for ubuntu-12.04 or ubuntu-16.04:

# Dockerfile for image cati/casa-dev:ubuntu-16.04

FROM cati/casa-test:ubuntu-16.04

USER root

# Install system dependencies
RUN apt-get install -y \
    build-essential \
    (...)
    liblapack-dev \
    <your_library> \  ###### HERE INSERT THE NAME OF THE EXTERNAL LIBRARY
  && apt-get clean

# Install Qt Installer Framework
COPY qt_installer_script /tmp/qt_installer_script
RUN wget -q http://download.qt.io/official_releases/qt-installer-framework/2.0.3/QtInstallerFramework-linux-x64.run -O /tmp/QtInstallerFramework-linux-x64.run && \
    chmod +x /tmp/QtInstallerFramework-linux-x64.run && \
    xvfb-run /tmp/QtInstallerFramework-linux-x64.run --script /tmp/qt_installer_script && \
    ln -s /usr/local/qt-installer/bin/* /usr/local/bin/ && \
    rm /tmp/QtInstallerFramework-linux-x64.run /tmp/qt_installer_script

(...)

###### OR WRITE THE COMMAND LINES TO INSTALL THE LIBRARY FROM SOURCES

USER brainvisa

After, run the script called create_images ([sources]/casa-distro/[trunk|bug_fix]/docker/create_images).

This script will rebuild casa-test and casa-dev images if the Dockefile was modified and will push all images in docker hub.

In our example, only the Dockerfile of casa-dev is different, so only casa-dev image will rebuilt.

The aim of a registry server for docker is to share private images of brainvisa for CATI members. .. Create the registry on https://catidev.cea.fr is more complicated due to CEA retrictions, so we use https://sandbox.brainvisa.info.

The Registry is compatible with Docker engine version 1.6.0 or higher.

In progress….

To update from changes in the image on server:

docker pull is208614:5000/casa/system

Troubleshooting¶

Typical problems are listed here.

System disk full¶

Docker images are big, and may grow bigger…

Change the filesystem / disk for docker images
cleanup leftover docker images or containers

Cannot build docker image, network access denied¶

With Docker versions older than 1.13, the docker build command did not have a host networking option. On some systems (Ubuntu 14 for instance) the contents of /etc/resolv.conf point to a local proxy DNS server (at least that’s what I understand), and docker could not use it during image building.

Either upgrade to a newer Docker, or change the DNS setup for Docker.

Cannot mount `~/.ssh/id_rsa` when starting docker¶

When docker starts, even when running as a specific user, it starts up as root. The mount options specified on docker commandline are setup as root. If the user home directory is on a network filesystem (NFS…), the local root user cannot override the filesystem access rights. Thus the directory tree must be traversable to reach the mounted directory.

In other words, the +x flag has to be set for “other” users on the directory and its parents. Typically:

chmod o+x ~
chmod o+x ~/.ssh

OpenGL is not working, or is slow¶

with docker¶

Several options are needed to enable display and OpenGL. Normally casa_distro tries to set them up and should do the best it can.

On machines with nvidia graphics cards and nvidia proprietary drivers, casa_distro will add options to mount the host system drivers and OpenGL libraries into the container in order to have hardware 3D rendering.

Options are setup in the casa_distro.json file so you can check and edit them. Therefore, the detection of nvidia drivers is done on the host machine at the time of build workflow creation: if the build workflow is shared accross several machines on a network, this config may not suit all machines running the container.

However it does not seem to work when ssh connections and remote display are involved.

with singularity¶

By default OpenGL runs using a software Mesa library. It should work in “most” cases, but in a slow manner. On machines with nvidia graphics cards and nvidia proprietary drivers, singularity has an option "--nv" to handle it. Casa-distro will try to detect nvidia drivers, and if they are found, will set this option. Contrarily to the docker case above, this detection is done each time a container is started, and the option is hard-coded in options passed by casa_distro to singularity (not editable in the config file).

There are cases where adding the nvidia option makes things worse (see ssh connections below). If you ever need to disable the nvidia option, you can add an option container_options=--no-nv to run, shell and other subcommands:

casa_distro run container_options=--no-nv glxinfo

If it is OK, you can set this option in the build workflow casa_distro.json config, under the "container_gui_env" key:

{
    "container_env": {
    # ...
    },
    "system": "ubuntu-16.04",
    "distro_source": "brainvisa",
    "container_gui_env": {
        "DISPLAY": "${DISPLAY}"
    },
    "container_mounts": {
    # ...
    },
    # ...
    "container_options": [
        "--pwd",
        "/casa/home"
    ],
    "container_gui_env": [
        "--no-nv"
    ],
    # ...
}

Via a ssh connection:

same host, different user:

xhost + must have been used on the host system. Works (as long as the XAUTHORITY env variable points to the .Xauthority file from the host user home directory).

different host:

I personally could not make it work using the --nv option. But actually outside of casa-distro or any container, it doesn’t work either. Remote GLX rendering has always been a very delicate thing…

It works for me using the software Mesa rendering (slow). So at this point, using casa_distro actually makes it possible to render OpenGL when the host system cannot (or not directly)…

Installation and setup¶

Setting up Docker¶

Install and use docker on an Ubuntu host¶

Change Docker’s storage base directory¶

Overview of the existing public brainvisa images¶

How to use a docker image¶

Cleaning up docker¶

Containers¶

Images¶

How to change the development environment ?¶

Troubleshooting¶

System disk full¶

Cannot build docker image, network access denied¶

Cannot mount `~/.ssh/id_rsa` when starting docker¶

OpenGL is not working, or is slow¶

with docker¶

with singularity¶

Table Of Contents

Previous topic

Installation and setup¶

Setting up Docker¶

Install and use docker on an Ubuntu host¶

Change Docker’s storage base directory¶

Overview of the existing public brainvisa images¶

How to use a docker image¶

Cleaning up docker¶

Containers¶

Images¶

How to change the development environment ?¶

Troubleshooting¶

System disk full¶

Cannot build docker image, network access denied¶

Cannot mount ~/.ssh/id_rsa when starting docker¶

OpenGL is not working, or is slow¶

with docker¶

with singularity¶

Cannot mount `~/.ssh/id_rsa` when starting docker¶