The casa_distro command

HELP, format: rst Casa_distro is the BrainVISA suite distribution swiss knife. It allows to setup a virtual environment and launch BrainVISA software. See http://brainivsa.info/casa-distro and https://github.com/brainvisa/casa-distro for more information

Version : 3.0.5

usage:

casa_distro <general options> <command> [<command parameters>...]
general optional arguments:
-v, --verbose Display as much information as possible.
--version Display casa-distro version number and exit.
-h, --help Display help message and exit. If used after command name, display only the help of this command.

Common parameters:

Most commands accept more or less the same parameters.

Many subcommands need environment selection specifications: see the documentation on how to specify an environment.

base_directory

default= $HOME/casa_distro

Directory where images and environments are stored. This parameter can be passed on the commandline, or set via the CASA_BASE_DIRECTORY environment variable.

Commands:

help

Print global help or help about a command.

Parameters

format
format help text in a given text format. Valid values are “text” (default) for raw text, or rst (RST/sphinx format).
full
if true or yes or 1, display each subcommand parameters documentation in the general help.

setup_user

Create a new user environment Parameters ———- distro

default=None Distro used to build this environment. This is typically “brainvisa”, “opensource” or “cati_platform”. Use “casa_distro distro” to list all currently available distro. Choosing a distro is mandatory to create a new environment. If the environment already exists, distro must be set only to reset configuration files to their default values.
version
version of the distro to use. By default the release with highest version is selected.
system
System to use inside this environment.
name
default={distro}-{version} Name of the environment. No other environment must have the same name (including developer environments). This name may be used later to select the environment to run.
container_type
default=None Type of virtual appliance to use. Either “singularity”, “vbox” or “docker”. If not given try to gues according to installed container software in the following order : Singularity, VirtualBox and Docker.
writable
size of a writable file system that can be used to make environement specific modification to the container file system. The size can be written in bytes as an integer, or in kilobytes with suffix “K”, or in megabytes qith suffix “M”, or in gygabytes with suffix “G”. If size is not 0, this will create an overlay.img file in the base environment directory. This file will contain the any modification done to the container file system.
base_directory

default= $HOME/casa_distro

Directory where images and environments are stored. This parameter can be passed on the commandline, or set via the CASA_BASE_DIRECTORY environment variable.

image
default={base_directory}/{distro}-{version}{extension} Location of the virtual image for this environement.
url
default=https://brainvisa.info/download URL where to download image if it is not found.
output
default={base_directory}/{name} Directory where the environement will be stored.
force
default=False force overwriting any existing matching environement. By default casa_distro will refuse to overwrite an existing one.
verbose

default=True

Print more detailed information if value is yes, true or 1.

setup_dev

Create a new developer environment Parameters ———- distro

default=opensource Distro used to build this environment. This is typically “brainvisa”, “opensource” or “cati_platform”. Use “casa_distro distro” to list all currently available distro. Choosing a distro is mandatory to create a new environment. If the environment already exists, distro must be set only to reset configuration files to their default values.
branch
default=master Name of the source branch to use for dev environments. Either “latest_release”, “master” or “integration”.
system
System to use with this environment. By default, it uses the first supported system of the selected distro.
image_version

default=1.0

If given, select environments having the given image version.

name
default={distro}-{branch}-{image_version} Name of the environment. No other environment must have the same name (including non developer environments). This name may be used later to select the environment to run.
container_type
default=None Type of virtual appliance to use. Either “singularity”, “vbox” or “docker”. If not given try to gues according to installed container software in the following order : Singularity, VirtualBox and Docker.
writable
size of a writable file system that can be used to make environement specific modification to the container file system. The size can be written in bytes as an integer, or in kilobytes with suffix “K”, or in megabytes qith suffix “M”, or in gygabytes with suffix “G”. If size is not 0, this will create an overlay.img file in the base environment directory. This file will contain the any modification done to the container file system.
base_directory

default= $HOME/casa_distro

Directory where images and environments are stored. This parameter can be passed on the commandline, or set via the CASA_BASE_DIRECTORY environment variable.

image
default={base_directory}/casa-dev-{image_version}{extension} Location of the virtual image for this environement.
url
default=https://brainvisa.info/download URL where to download image if it is not found.
output
default={base_directory}/{name} Directory where the environement will be stored.
force
default=False force overwriting any existing matching environement. By default casa_distro will refuse to overwrite an existing one.
verbose

default=True

Print more detailed information if value is yes, true or 1.

distro

List all available distro and provide information for each one.

list

List run or dev environments created by “setup”/”setup_dev” command.

Parameters

type

default=None

If given, select environment having the given type.

distro

default=None

If given, select environment having the given distro name.

branch

default=None

If given, select environment having the given branch.

system

default=None

If given, select environments having the given system name.

image_version

default=None

If given, select environments having the given image version.

name

default=None

If given, select environment by its name. It replaces type, distro, branch and system and is shorter to select one.

base_directory

default= $HOME/casa_distro

Directory where images and environments are stored. This parameter can be passed on the commandline, or set via the CASA_BASE_DIRECTORY environment variable.

json
default = no The output is written as a list of configuration dictionaries in JSON format.
verbose

default=None

Print more detailed information if value is yes, true or 1.

run

Start any command in a selected run or dev environment

example:

casa_distro branch=master ls -als /casa

Parameters

type

default=None

If given, select environment having the given type.

distro

default=None

If given, select environment having the given distro name.

branch

default=None

If given, select environment having the given branch.

system

default=None

If given, select environments having the given system name.

image_version

default=None

If given, select environments having the given image version.

name

default=None

If given, select environment by its name. It replaces type, distro, branch and system and is shorter to select one.

version
If given, select environment by its version (only applicable to user environments, not dev)
base_directory

default= $HOME/casa_distro

Directory where images and environments are stored. This parameter can be passed on the commandline, or set via the CASA_BASE_DIRECTORY environment variable.

gui

default=True

If no, false or 0, command is not using a graphical user interface (GUI). Nothing is done to connect the container to a graphical interface. This option may be necessary in context where a graphical interface is not available.

opengl

default=auto

Setup different ways of trying to use OpenGL 3D rendering and GPU. auto, container, nv, or software.

  • auto: use a heuristic to choose the best option that is safe based on the host configuration
  • container: passes no special options to Singularity: the mesa installed in the container is used
  • nv tries to mount the proprietary NVidia driver of the host (linux) system in the container
  • software sets LD_LIBRARY_PATH to use a software-only OpenGL rendering. This solution is the slowest but is a fallback when no other solution works.
root

default=False

If yes, true or 1, start execution as system administrator. For Singularity container, this requires administrator privileges on host system.

cwd
Set current working directory to the given value before launching the command. By default, it is the same working directory as on the host
env
Comma separated list of environment variables to pass to the command. Each variable must have the form name=value.
image
Force usage of a specific virtual image instead of the one defined in the environment configuration.
container_options
Comma separated list of options to add to the command line used to call the container system.
verbose

default=None

Print more detailed information if value is yes, true or 1.

pull_image

Update the container images. By default all images that are used by at least one environment are updated. There are two ways of selecting the image(s) to be downloaded:

  1. filtered by environment, using the name selector, or a combination of distro, branch, and image_version.

    casa_distro pull_image type=dev image_version=5.0

  2. directly specifying an image name (not including bulild_number), e.g.:

    casa_distro pull_image image=casa-run-5.0.sif

Note that even though the build number does not appear in the local image name, this command will always pull the image with the highest build_number, e.g. the image that is named casa-dev-5.0-1.sif on the server will be downloaded to a file named casa-dev-5.0.sif.

Parameters

distro

default=None

If given, select environment having the given distro name.

branch

default=None

If given, select environment having the given branch.

system

default=None

If given, select environments having the given system name.

type

default=None

If given, select environment having the given type.

image_version

default=None

If given, select environments having the given image version.

version
If given, select environment by its version (only applicable to user environments, not dev)
name

default=None

If given, select environment by its name. It replaces type, distro, branch and system and is shorter to select one.

base_directory

default= $HOME/casa_distro

Directory where images and environments are stored. This parameter can be passed on the commandline, or set via the CASA_BASE_DIRECTORY environment variable.

image
Force usage of a specific virtual image instead of the one defined in the environment configuration.
url
default=https://brainvisa.info/download URL where to download image if it is not found.
force
default=False force re-download of images even if they are locally present and up-to-date.
verbose

default=None

Print more detailed information if value is yes, true or 1.

list_images

List the locally installed container images. There are two ways of selecting the image(s):

  1. filtered by environment, using the name selector, or a combination of distro, branch, and system.

  2. directly specifying a full image name, e.g.:

    casa_distro list_image image=casa-run-ubuntu-18.04.sif

Parameters

distro

default=None

If given, select environment having the given distro name.

branch

default=None

If given, select environment having the given branch.

system

default=None

If given, select environments having the given system name.

image_version

default=None

If given, select environments having the given image version.

version
If given, select environment by its version (only applicable to user environments, not dev)
name

default=None

If given, select environment by its name. It replaces type, distro, branch and system and is shorter to select one.

type

default=None

If given, select environment having the given type.

image
Force usage of a specific virtual image instead of the one defined in the environment configuration.
base_directory

default= $HOME/casa_distro

Directory where images and environments are stored. This parameter can be passed on the commandline, or set via the CASA_BASE_DIRECTORY environment variable.

verbose

default=None

Print more detailed information if value is yes, true or 1.

shell

Start a bash shell in the configured container with the given pository configuration.

Parameters

type

default=None

If given, select environment having the given type.

distro

default=None

If given, select environment having the given distro name.

branch

default=None

If given, select environment having the given branch.

system

default=None

If given, select environments having the given system name.

image_version

default=None

If given, select environments having the given image version.

name

default=None

If given, select environment by its name. It replaces type, distro, branch and system and is shorter to select one.

version
If given, select environment by its version (only applicable to user environments, not dev)
base_directory

default= $HOME/casa_distro

Directory where images and environments are stored. This parameter can be passed on the commandline, or set via the CASA_BASE_DIRECTORY environment variable.

gui

default=True

If no, false or 0, command is not using a graphical user interface (GUI). Nothing is done to connect the container to a graphical interface. This option may be necessary in context where a graphical interface is not available.

opengl

default=auto

Setup different ways of trying to use OpenGL 3D rendering and GPU. auto, container, nv, or software.

  • auto: use a heuristic to choose the best option that is safe based on the host configuration
  • container: passes no special options to Singularity: the mesa installed in the container is used
  • nv tries to mount the proprietary NVidia driver of the host (linux) system in the container
  • software sets LD_LIBRARY_PATH to use a software-only OpenGL rendering. This solution is the slowest but is a fallback when no other solution works.
root

default=False

If yes, true or 1, start execution as system administrator. For Singularity container, this requires administrator privileges on host system.

cwd
Set current working directory to the given value before launching the command. By default, it is the same working directory as on the host
env
Comma separated list of environment variables to pass to the command. Each variable must have the form name=value.
image
Force usage of a specific virtual image instead of the one defined in the environment configuration.
container_options
Comma separated list of options to add to the command line used to call the container system.
verbose

default=None

Print more detailed information if value is yes, true or 1.

mrun

Start any command in one or several container with the given repository configuration. By default, command is executed in all existing build workflows.

example:

# Launch bv_maker on all build workflows using any version of Ubuntu
casa_distro mrun bv_maker system=ubuntu-*

Parameters

type

default=None

If given, select environment having the given type.

distro

default=None

If given, select environment having the given distro name.

branch

default=None

If given, select environment having the given branch.

system

default=None

If given, select environments having the given system name.

image_version

default=None

If given, select environments having the given image version.

name

default=None

If given, select environment by its name. It replaces type, distro, branch and system and is shorter to select one.

version
If given, select environment by its version (only applicable to user environments, not dev)
base_directory

default= $HOME/casa_distro

Directory where images and environments are stored. This parameter can be passed on the commandline, or set via the CASA_BASE_DIRECTORY environment variable.

gui

default=True

If no, false or 0, command is not using a graphical user interface (GUI). Nothing is done to connect the container to a graphical interface. This option may be necessary in context where a graphical interface is not available.

opengl

default=auto

Setup different ways of trying to use OpenGL 3D rendering and GPU. auto, container, nv, or software.

  • auto: use a heuristic to choose the best option that is safe based on the host configuration
  • container: passes no special options to Singularity: the mesa installed in the container is used
  • nv tries to mount the proprietary NVidia driver of the host (linux) system in the container
  • software sets LD_LIBRARY_PATH to use a software-only OpenGL rendering. This solution is the slowest but is a fallback when no other solution works.
root

default=False

If yes, true or 1, start execution as system administrator. For Singularity container, this requires administrator privileges on host system.

cwd
Set current working directory to the given value before launching the command. By default, it is the same working directory as on the host
env
Comma separated list of environment variables to pass to the command. Each variable must have the form name=value.
image
Force usage of a specific virtual image instead of the one defined in the environment configuration.
container_options
Comma separated list of options to add to the command line used to call the container system.
verbose

default=None

Print more detailed information if value is yes, true or 1.

bv_maker

Start a bv_maker in the configured container with the given repository configuration.

Parameters

type

default=dev

If given, select environment having the given type.

distro

default=None

If given, select environment having the given distro name.

branch

default=None

If given, select environment having the given branch.

system

default=None

If given, select environments having the given system name.

image_version

default=None

If given, select environments having the given image version.

name

default=None

If given, select environment by its name. It replaces type, distro, branch and system and is shorter to select one.

base_directory

default= $HOME/casa_distro

Directory where images and environments are stored. This parameter can be passed on the commandline, or set via the CASA_BASE_DIRECTORY environment variable.

env
Comma separated list of environment variables to pass to the command. Each variable must have the form name=value.
image
Force usage of a specific virtual image instead of the one defined in the environment configuration.
container_options
Comma separated list of options to add to the command line used to call the container system.
verbose

default=None

Print more detailed information if value is yes, true or 1.

clean_images

Delete singularity images which are no longer used in any build workflow, or those listed in the “image” parameter. There are two ways of selecting the image(s):

  1. filtered by environment, using the name selector, or a combination of distro, branch, and system.

  2. directly specifying a full image name, e.g.:

    casa_distro clean_images image=casa-run-ubuntu-18.04.sif

Parameters

distro

default=None

If given, select environment having the given distro name.

branch

default=None

If given, select environment having the given branch.

system

default=None

If given, select environments having the given system name.

image_version

default=None

If given, select environments having the given image version.

version
If given, select environment by its version (only applicable to user environments, not dev)
name

default=None

If given, select environment by its name. It replaces type, distro, branch and system and is shorter to select one.

type

default=None

If given, select environment having the given type.

image
Force usage of a specific virtual image instead of the one defined in the environment configuration.
base_directory

default= $HOME/casa_distro

Directory where images and environments are stored. This parameter can be passed on the commandline, or set via the CASA_BASE_DIRECTORY environment variable.

interactive
default=True ask confirmation before deleting an image
verbose

default=False

Print more detailed information if value is yes, true or 1.

delete

Delete an existing environment.

The whole environment directory will be removed and forgotten.

Use with care.

Image files will be left untouched - use clean_images for this.

Parameters

type

default=None

If given, select environment having the given type.

distro

default=None

If given, select environment having the given distro name.

branch

default=None

If given, select environment having the given branch.

system

default=None

If given, select environments having the given system name.

image_version

default=None

If given, select environments having the given image version.

version
If given, select environment by its version (only applicable to user environments, not dev)
name

default=None

If given, select environment by its name. It replaces type, distro, branch and system and is shorter to select one.

base_directory

default= $HOME/casa_distro

Directory where images and environments are stored. This parameter can be passed on the commandline, or set via the CASA_BASE_DIRECTORY environment variable.

interactive
default=True if true (or 1, or yes), ask confirmation interactively for each selected environement.

The complexity of arguments parsing

As casa_distro runs a chain of sub-commands, each of them passing some user-provided arguments, there may be some ambiguity on who should receive arguments. The argument delimiter -- should be used appropriately. For instance:

casa_distro -r ~/casa_distro run branch=bug_fix gui=1 anatomist /casa/tests/some_image.nii

Casa_distro will run the anatomist command through Docker or Singularity, which also parses its commandline options:

  • Casa_distro gets arguments run, branch=bug_fix, and gui=1, and identifies the remaining args as the command to run.
  • It invokes docker or singularity, passing it some specific arguments, and the remaining anatomist and /casa/tests/some_image.nii, which will be executed in a bash shell inside docker.

But the following command is ambiguous:

casa_distro -r ~/casa_distro run branch=bug_fix bv_maker gui=1 --directory=/tmp

Why ?

The last argument, --directory=/tmp, will be interpreted by casa_distro, and you will get an error from the casa_distro command which does not know this option (but even if it did, it would not pass it to the bv_maker command). But as it is located, you likeky expected to get passed to bv_maker… At the moment casa_distro intercetps all arguments in the shape argument=value for himself.

In the first example there was not this ambiguity since casa_distro did not recognize the arguments anatomist and /casa/tests/some_image.nii, so passed them to the following (docker or singularity) command. But now, --directory=/tmp is understood by casa_distro, used, and consumed by it.

So, how to pass the option to bv_maker ?

You have to stop arguments parsing inside casa_distro and make it pass the remaining arguments to the following command, using --:

casa_distro -r ~/casa_distro run branch=bug_fix gui=1 -- bv_maker --directory=/tmp

The other useful option is to pass options to the container program (docker for instance) (not to the command executed inside docker), typically to mount host directories etc. This is done using the container_optioins option, followed by more options:

casa_distro -r ~/casa_distro run branch=bug_fix gui=1 container_options='-v /home/albert/my_data:/docker_data' -- anatomist /docker_data/image.nii

Options common to several commands

Environment specification

distro=brainvisa
branch=bug_fix
system=ubuntu-16.04

An environment may also be given as its unique name insead of the above variables.

Environment variables

Rather than systematically passing options, some environment variables may be used to specify some parameters to ̀ casa_distro`:

# replaces the -r option
CASA_DEFAULT_REPOSITORY=/home/someone/casa_distro

Environment configuration file

The casa_distro.json file found in each environment subdirectory (in the conf subdirectory, actually) is a dictionary which contains varaibles used to define the environment, the type of container used (docker or singularity), mounted directories in the container image, etc.

Some variables substitution can occur in the string values, in a “pythonic” shape: %(variable)s will be replaced by the contents of a variable variable. The following variables are available:

name
casa_distro_compatibility
image
system
container_type
branch
mounts
type
distro

Moreover some environment variables replacement also takes place, in the shape: ${VARIABLE}.

Configuration dictionary variables

branch: string
name of the source and build branch (master, latest_release, release_candidate)
container_env: dictionary
environment variables set when running a container.
container_gui_env: dictionary
environment variables set when running a container in gui mode.
container_gui_options: list
list of commandline options passed to the container command in gui mode: depends on the container types.
image: string
container image name. May be a filename, or a docker-style identifier. Docker-style identifiers are converted into filenames when using singularity, thus are still understood, so cati/casa-dev:ubuntu-16.04 is a valid name.
container_options: list
list of commandline options passed to the container command: depends on the container types, options passed to docker and to singularity actually differ.
container_type: string
docker or singularity. New container types, virtualbox for instance, may be added in future extensions.
mounts: dictionary
mount points in the container. Directories from the host filesystem (source) are exported to the container (dest). The dictionary is a map of destination:source directories.
distro: string
name of the distribution (set of configured sources built in the build workflow).
system: string
system the container runs (ubuntu-18.04, etc.).