Using the casa_distro_admin command¶
casa_distro_admin
command is the part of casa-distro which is not needed for regular users / developers: it allows to create docker and singularity system images, push them on a server to be downloadable by regular users, and to manage brainisa distributions release plans.
The admin part has been separated from the other casa_distro
command, both for sake of clarity, and because the admin part is bigger since it contains everything needed to build system images, including some precompiled software for various operating systems, some libraries source code, etc. All this makes the casa-distro complete packages more than 100MB large, whereas the user part is only 300 or 400 KB zipped.
Subcommands¶
help¶
package_casa_distro¶
publish_casa_distro¶
create_release_plan¶
update_release_plan¶
publish_release_plan¶
apply_release_plan¶
create_docker¶
update_docker¶
publish_docker¶
create_singularity¶
create_singularity converts a docker image into a singularity image. To do so it needs to unpack the docker image into a temporary directory, and re-pack it in a different way. It requires
- sudo / root privileges: the user has to be allowed to run sodo commands, the password will be asked when needed.
- lots of temporary disk space. It may be needed to setup the following environment variables while running
casa_distro_admin create_singularity
: TMPDIR
SINGULARITY_TMPDIR
SINGULARITY_CACHEDIR
- lots of temporary disk space. It may be needed to setup the following environment variables while running
And create_docker needs to have run before, or the docker image should have been pulled. Thus docker is required at this point in the current way things have been impemented (this may change in the future).
publish_singularity¶
publish_build_workflows¶
Managing container images¶
Singularity and Docker¶
In a general way, we designed casa-distro first using Docker, and then extended it to Singularity. We use Singularity by actually converting Docker images, so images should be built first using Docker. So Docker is needed to build Singularity images for Casa-distro (this is not needed for regular users, which just download system images, and can use singularity alone, or docker alone).
Thus, to build a singularity image, one needs to use first casa_distro_admin create_docker
, then casa_distro_admin create_singularity
.
Sharing images¶
Images can be uploaded to a centralized place to allow other users to use them.
By default publish_docker
will use Docker Hub, which requires an account and access rights to push images.
Singularity did not have a public hub by the time we started casa-distro, so publish_singularity
will push by default somewhere on the BrainVISA web site, which also requires to have access to the website account to push. We could use Singularity Hub in the future.
Downloading and updating images do not need any access rights, they are public.
Designing new images¶
create_docker may be used to update and rebuild the images which are proposed inside the casa-distro distribution, but this specific use has a limited benefit since they are normally pre-built and are downloaded from the hubs when needed.
create_docker may prove useful however for an application designer who needs to ship specific or additional system packages in the images. In this situation a new image type needs to be designed.
New image types correspond to subdirectories which will looked for in the following directory trees (when they exist):
<workflow_repository>/share/docker/<image_name>/<system>
$HOME/.config/casa-distro/docker/<image_name>/<system>
$HOME/.casa-distro/docker/<image_name>/<system>
<casa_builtin>/docker/<image_name>/<system>
where:
<workflow_repository>
is the main workflow repository either passed via the--repository
option, or via theCASA_DEFAULT_REPOSITORY
environment variable, or in the default location$HOME/casa_distro
<casa_builtin>
if the builtin share directory of casa-distro (or its sources)<image_name>
is a name (or type name) for the image, like the builtin onescasa-test
,casa-dev
etc.<system>
is the name of the system running inside the docker image (the builtin ones areubuntu-18.04
,ubuntu-16.04
,ubuntu-14.04
,ubuntu-12.04
,centos-7.4
,windows-7-32
,windows-7-64
.
So custom, user-defined images can be added in a personal directory. Such an image definition directory should contain at least two files:
casa_distro_docker.yaml
is a Yaml file definig dependencies, name and tags for the image. Ex:
dependencies:
- ../../casa-dev
image_sources:
- name: pytorch
tags:
- ubuntu-16.04
visibility: public
- a Dockerfile The Dockerfile may (should) be based on another image, in the usual way of building docker images. Thus an existing casa-distro image can be the base for a new one.
Once an image is created with docker, it can be converted to singularity using casa_distro create_singularity
.