This remote module of the [Orfeo ToolBox](https://www.orfeo-toolbox.org) provides a generic, multi purpose deep learning framework, targeting remote sensing images processing.
It contains a set of new process objects that internally invoke [Tensorflow](https://www.tensorflow.org/), and a bunch of user-oriented applications to perform deep learning with real-world remote sensing images.
Applications can be used to build OTB pipelines from Python or C++ APIs.
### Highlights
- Sampling,
- Training, supporting save/restore/import operations (a model can be trained from scratch or fine-tuned),
- Serving models with support of OTB streaming mechanism. Meaning (1) not limited by images sizes, (2) can be used as a "lego" in any OTB pipeline and preserve streaming, (3) MPI support available (use multiple processing unit to generate one single output image)
## Features
### OTB Applications
- Sample patches in remote sensing images with `PatchesExtraction`,
- Model training, supporting save/restore/import operations (a model can be trained from scratch or fine-tuned) with `TensorflowModelTrain`,
- Inference with support of OTB streaming mechanism with `TensorflowModelServe`. The streaming mechanism means (1) no limitation with images sizes, (2) inference can be used as a "lego" in any OTB pipeline (using C++ or Python APIs) and preserving streaming, (3) MPI support available (use multiple processing unit to generate one single output image)
### Python API
This is a work in progress. For now, `tricks.py` provides a set of helpers to build deep nets, and `otbtf.py` provides datasets which can be used in Tensorflow pipelines to train networks from python.
### Portfolio
## Portfolio
Below are some screen captures of deep learning applications performed at large scale with OTBTF.
- Image to image translation (Spot-7 image --> Wikimedia Map using CGAN)
...
...
@@ -31,7 +36,7 @@ You can read more details about these applications on [this blog](https://mdl4eo
For now you have two options: either use the existing **docker image**, or build everything yourself **from source**.
(You can also find plenty of OTBTF flavored images [here](https://gitlab.com/latelescop/docker/otbtf/container_registry/)).
All GPU docker images are suited for **NVIDIA GPUs**.
They use CUDA/CUDNN support and are built with compute capabilities 5.2, 6.1, 7.0, 7.5.
To change the compute capabilities, you can build your own docker image using the provided dockerfile. See the [docker build documentation} (tools/dockerfiles).
You can find more details on the **GPU docker image** and some **docker tips and tricks** on [this blog](https://mdl4eo.irstea.fr/2019/10/15/otbtf-docker-image-with-gpu/).
Also you can check [this document](https://gitlab.irstea.fr/raffaele.gaetano/moringa/-/tree/develop/docker) that also mentions useful stuff.
Read more in the [docker use documentation](doc/DOCKERUSE.md).
### Build from sources
See [here](doc/HOWTOBUILD.md) to see how to build the remote module from sources.
Read more in the [build from sources documentation](doc/HOWTOBUILD.md).
## How to use it?
## How to use
- Reading [the documentation](doc/APPLICATIONS.md) will help, of course 😉
- Reading [the applications documentation](doc/APPLICATIONS.md) will help, of course 😉
- A small [tutorial](https://mdl4eo.irstea.fr/2019/01/04/an-introduction-to-deep-learning-on-remote-sensing-images-tutorial/) on MDL4EO's blog
- in the `python` folder are provided some [ready-to-use deep networks, with documentation and scientific references](doc/EXAMPLES.md).
- A book: *Cresson, R. (2020). Deep Learning for Remote Sensing Images with Open Source Software. CRC Press.* Use QGIS, OTB and Tensorflow to perform various kind of deep learning sorcery on remote sensing images (patch-based classification for landcover mapping, semantic segmentation of buildings, optical image restoration from joint SAR/Optical time series).
