# Docker multi-stage build with external bazel cache
# Build with Docker
Docker build has to be called from the root of the repository (i.e. `docker build .` or `bash tools/docker/multibuild.sh`).
Docker build has to be called from the root of the repository (i.e. `docker build .` or `bash tools/docker/multibuild.sh`).
You may build a custom docker image using `--build-arg` and the config files in this directory.
You can build a custom image using `--build-arg` and several config files :
For TensorFlow, see the `TF` arg for the git branch/tag, [build-env-tf.sh](build-env-tf.sh) and BZL_* arguments for the build configuration.
- Ubuntu : `BASE_IMG` should accept any version, for additional packages see [build-deps-cli.txt](build-deps-cli.txt) and [build-deps-gui.txt](build-deps-gui.txt)
Regarding OTB, you can edit cmake flags in [build-flags-otb.txt](build-flags-otb.txt) and the `OTB` argument for the git branch/tag to clone.
- TensorFlow : `TF` arg for the git branch or tag + [build-env-tf.sh](build-env-tf.sh) and BZL_* arguments for the build configuration
If you need additional Ubuntu packages see [build-deps-cli.txt](build-deps-cli.txt) and [build-deps-gui.txt](build-deps-gui.txt) for GUI related packages - it is disabled by default in order to save space, and because docker xvfb isn't working properly with opengl.
- OrfeoToolBox : `OTB` arg for the git branch or tag + [build-flags-otb.txt](build-flags-otb.txt) to edit cmake flags
There is no way make a common build of OTB shared between docker builds, since we're using different BASE_IMG and because of the multi-stage Dockerfile.
If you just need to rebuild with different GUI or KEEP_SRC arguments, or may be a different branch of OTB, bazel cache will help you to rebuild everything except TF, even if the docker cache was purged (after `docker [system|builder] prune`).
But if you just need to rebuild with different GUI or KEEP_SRC arguments, or may be a different branch of OTB, bazel cache may help you to rebuild everything except TF, even if docker cache was purged (after `docker system prune`).
In order to recycle the cache, bazel config and TF git tag should be exactly the same, any change in [build-env-tf.sh](build-env-tf.sh) and `--build-arg` (if related to bazel env, cuda, mkl, xla...) may result in a fresh new build.
In order to recycle the cache, bazel config and TF git tag has to be exactly the same, any change in [build-env-tf.sh](build-env-tf.sh) and `--build-arg` (if related to bazel env, cuda, mkl, xla...) may result in a complete new build.
Start a cache daemon - here with max 20GB but 12GB should be enough to save 2 TF builds (GPU and CPU):
Start a cache daemon - here with max 20GB but 12GB should be enough to save 2 TF builds (GPU and CPU):
```bash
```bash
mkdir-p$HOME/.cache/bazel-remote
mkdir-p$HOME/.cache/bazel-remote
docker run --detach-u 1000:1000 -v$HOME/.cache/bazel-remote:/data -p 9090:8080 buchgr/bazel-remote-cache --max_size=20
docker run --detach-u 1000:1000 -v$HOME/.cache/bazel-remote:/data -p 9090:8080 buchgr/bazel-remote-cache --max_size=20
```
```
Then just add ` --network='host'` to the docker build command, or connect bazel to another adress (see the 'BZL_OPTIONS' build argument - the other way of docker is a virtual bridge, you'll need to edit the IP address).
Then just add ` --network='host'` to the docker build command, or connect bazel to a remote server - see 'BZL_OPTIONS'.
The other way of docker is a virtual bridge, but you'll need to edit the IP address.
## Build examples
## Build examples
```bash
```bash
# Build for CPU using default Dockerfiles args (without AWS, HDFS and GCP support)
# Build for CPU using default Dockerfiles args (without AWS, HDFS or GCP support)
# You may edit the Dockerfile to clone an old branch of the repo instead of copying files from the build context
# You could edit the Dockerfile in order to clone an old branch of the repo instead of copying files from the build context
# Numpy version requirement :
# TF < 2.4 ==> numpy<1.19.0,>=1.16.0
# TF >= 2.4 ==> numpy~=1.19
```
```
### Debug build
### Debug build
If you fail to build, you can log into the last layer and check CMake logs.
If you fail to build, you can log into the last layer and check CMake logs. Run `docker images`, find the latest layer ID and run a tmp container (`docker run -it d60496d9612e bash`).
Run `docker images`, find the latest layer ID and run a tmp container (`docker run -it d60496d9612e bash`).
You may also need to split some multi-command layers in the Dockerfile.
You may also need to split some multi-command layers in the Dockerfile.
If you see OOM errors during SuperBuild you should decrease CPU_RATIO (e.g. 0.75).
If you see OOM errors during SuperBuild you should decrease CPU_RATIO (e.g. 0.75).
## Container examples
## Container examples
```bash
```bash
# Pull GPU image and create container with your home directory as volume
# Pull GPU image and create a new container with your home directory as volume (requires apt package nvidia-docker2 and CUDA>=11.0)
# (requires apt package nvidia-docker2 and CUDA>=11.0)
