README.md 2.87 KB
Newer Older
Matthieu Boileau's avatar
Matthieu Boileau committed
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45
# Overview

This a set of example bash scripts designed to perform repeatable and cumulative installations on a HPC cluster where library versions are handled by [environment modules](http://modules.sourceforge.net/).

Two assumptions:

- libraries are installed in a shared partition
- module files are named according to the following scheme: `{library_version}_{compiler_version}_{main_dependency_version}`

Example of resulting module organization:

```bash
tree modules/tools/
modules/tools/
├── cmake
│   └── 3.9.2_gcc640
├── paraview
│   ├── 5.3.0_gcc640_openmpi1107
│   ├── 5.3.0_gcc640_openmpi211
│   ├── 5.4.1_gcc640_openmpi1107
│   ├── 5.4.1_gcc640_openmpi211
│   └── 5.4.1-py3_gcc640_openmpi1107
└── singularity
    └── 2.4.0_gcc640
```

and corresponding installation directories (here for `paraview`):

```bash
tree -L 3 install/paraview/
install/paraview/
├── 5.3.0
│   └── gcc-6.4.0
│       ├── openmpi-1.10.7
│       └── openmpi-2.1.1
├── 5.4.1
│   └── gcc-6.4.0
│       ├── openmpi-1.10.7
│       └── openmpi-2.1.1
└── 5.4.1-py3
    └── gcc-6.4.0
        └── openmpi-1.10.7
```


Matthieu Boileau's avatar
Matthieu Boileau committed
46 47
# Installation

Matthieu Boileau's avatar
Matthieu Boileau committed
48 49 50
## Dependencies

Requires only `bash` for execution and `envtpl` for module file templating:
Matthieu Boileau's avatar
Matthieu Boileau committed
51 52 53 54 55

```
pip install envtpl
```

Matthieu Boileau's avatar
Matthieu Boileau committed
56 57 58 59 60 61 62 63 64 65 66 67
## Configuration

Set your custom paths in `common.sh` for the 3 directories:

```bash
PREWORK_DIR=${SOFT_DIR}/sources  # sources download and compilation top directory
PREINSTALL_DIR=${SOFT_DIR}/install  # libraries installation top directory
PREMODULE_DIR=${SOFT_DIR}/modules  # modulefiles top directory
```

These directories will be created if they don't exist.

Matthieu Boileau's avatar
Matthieu Boileau committed
68 69
# Usage

Matthieu Boileau's avatar
Matthieu Boileau committed
70 71
Each sub directory of this project corresponds to a library (or application) installation.
For most cases, it contains 2 files:
Matthieu Boileau's avatar
Matthieu Boileau committed
72 73 74

- `module.tmpl`, a module file templated with Jinja2
- `install.sh`, a bash script to be run:
Matthieu Boileau's avatar
Matthieu Boileau committed
75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111
	- `./install.sh` without argument
		1. download && compile && install the library
		2. build && install the module file
	- `./install.sh clean`: remove the temporary directories used for download and compilation
	- `./install.sh module`: build && install only the module file

## Create a new library installation

- Copy an existing directory and adapt the `install.sh` and `module.tmpl` files.
- From this directory, run:

```bash
./install.sh
```

## Install a new version of an existing library

- `install.sh`: the only parameters values that may be changed are located at the top of the file.
For example for [paraview/install.sh](paraview/install.sh):

```bash
LIB_NAME="paraview"  # unchanged
LIB_VERSION_URL=5.4.1
LIB_VERSION=${LIB_VERSION_URL}-py3
GCC_VERSION=6.4.0
MPI_LIB=openmpi
MPI_VERSION=1.10.7
HDF5_VERSION=1.10.1
CMAKE_VERSION=3.9.2
```

- `module.tmpl` remains unchanged.
- Run again:

```
./install.sh
```