Commit 70004b8e authored by Cresson Remi's avatar Cresson Remi
Browse files

Merge branch 'develop'

parents 5c0ec1a9 4d9c113a
......@@ -16,7 +16,7 @@ This remote module has been tested successfully on Ubuntu 16.04 and CentOs 7 wit
First, **build the latest *develop* branch of OTB from sources**. You can check the [OTB documentation](https://www.orfeo-toolbox.org/SoftwareGuide/SoftwareGuidech2.html) which details all the steps, if fact it is quite easy thank to the SuperBuild.
## Build TensorFlow
Then you have to **build Tensorflow from source** except if you want to use only the sampling applications of OTBTensorflow (in this case, skip this section).
Then you have to **build Tensorflow from source** except if you want to use only the sampling applications of OTBTensorflow (in this case, skip this section).
Follow [the instructions](https://www.tensorflow.org/install/install_sources) to build Tensorflow.
## Build this remote module
......@@ -71,11 +71,11 @@ However, you can use any deep net available on the web, or use an existing gui a
The important thing here is to know the following parameters for your **placeholders** (the inputs of your model) and **output tensors** (the outputs of your model).
- For each **input placeholder**:
-- Name
-- Perceptive field
-- Receptive field
- For each **output tensor**:
--Name
--Expression field
--Scale factor
-- Name
-- Expression field
-- Scale factor
![Schema](doc/schema.png)
......@@ -89,47 +89,47 @@ Set you input data for training and for validation. The validation against test
![Schema](doc/model_training.png)
```
This is the (TensorflowModelTrain) application, version 6.5.0
Train a multisource deep learning net using Tensorflow. Change the OTB_TF_NSOURCES environment variable to set the number of sources.
Parameters:
-model <group> Model parameters
MISSING -model.dir <string> Tensorflow model_save directory (mandatory)
-model.restorefrom <string> Restore model from path (optional, off by default)
-model.saveto <string> Save model to path (optional, off by default)
-training <group> Training parameters
-training.batchsize <int32> Batch size (mandatory, default value is 100)
-training.epochs <int32> Number of epochs (mandatory, default value is 10)
-training.userplaceholders <string list> Additional single-valued placeholders for training. Supported types: int, float, bool. (optional, off by default)
MISSING -training.targetnodesnames <string list> Names of the target nodes (mandatory)
-training.outputtensorsnames <string list> Names of the output tensors to display (optional, off by default)
-training.source1 <group> Parameters for source #1 (training)
MISSING -training.source1.il <string list> Input image (or list to stack) for source #1 (training) (mandatory)
MISSING -training.source1.fovx <int32> Field of view width for source #1 (mandatory)
MISSING -training.source1.fovy <int32> Field of view height for source #1 (mandatory)
MISSING -training.source1.placeholder <string> Name of the input placeholder for source #1 (training) (mandatory)
-training.source2 <group> Parameters for source #2 (training)
MISSING -training.source2.il <string list> Input image (or list to stack) for source #2 (training) (mandatory)
MISSING -training.source2.fovx <int32> Field of view width for source #2 (mandatory)
MISSING -training.source2.fovy <int32> Field of view height for source #2 (mandatory)
MISSING -training.source2.placeholder <string> Name of the input placeholder for source #2 (training) (mandatory)
-validation <group> Validation parameters
-validation.mode <string> Metrics to compute [none/class/rmse] (mandatory, default value is none)
-validation.userplaceholders <string list> Additional single-valued placeholders for validation. Supported types: int, float, bool. (optional, off by default)
-validation.source1 <group> Parameters for source #1 (validation)
-validation.source1.il <string list> Input image (or list to stack) for source #1 (validation) (mandatory)
-validation.source1.placeholder <string> Name of the input placeholder for source #1 (validation) (mandatory)
-validation.source2 <group> Parameters for source #2 (validation)
-validation.source2.il <string list> Input image (or list to stack) for source #2 (validation) (mandatory)
-validation.source2.placeholder <string> Name of the input placeholder for source #2 (validation) (mandatory)
-inxml <string> Load otb application from xml file (optional, off by default)
-progress <boolean> Report progress
-help <string list> Display long help (empty list), or help for given parameters keys
-model <group> Model parameters
MISSING -model.dir <string> Tensorflow model_save directory (mandatory)
-model.restorefrom <string> Restore model from path (optional, off by default)
-model.saveto <string> Save model to path (optional, off by default)
-training <group> Training parameters
-training.batchsize <int32> Batch size (mandatory, default value is 100)
-training.epochs <int32> Number of epochs (mandatory, default value is 10)
-training.userplaceholders <string list> Additional single-valued placeholders for training. Supported types: int, float, bool. (optional, off by default)
MISSING -training.targetnodes <string list> Names of the target nodes (mandatory)
-training.outputtensors <string list> Names of the output tensors to display (optional, off by default)
-training.usestreaming <boolean> Use the streaming through patches (slower but can process big dataset) (optional, off by default, default value is false)
-training.source1 <group> Parameters for source #1 (training)
MISSING -training.source1.il <string list> Input image (or list to stack) for source #1 (training) (mandatory)
MISSING -training.source1.patchsizex <int32> Patch size (x) for source #1 (mandatory)
MISSING -training.source1.patchsizey <int32> Patch size (y) for source #1 (mandatory)
MISSING -training.source1.placeholder <string> Name of the input placeholder for source #1 (training) (mandatory)
-training.source2 <group> Parameters for source #2 (training)
MISSING -training.source2.il <string list> Input image (or list to stack) for source #2 (training) (mandatory)
MISSING -training.source2.patchsizex <int32> Patch size (x) for source #2 (mandatory)
MISSING -training.source2.patchsizey <int32> Patch size (y) for source #2 (mandatory)
MISSING -training.source2.placeholder <string> Name of the input placeholder for source #2 (training) (mandatory)
-validation <group> Validation parameters
-validation.mode <string> Metrics to compute [none/class/rmse] (mandatory, default value is none)
-validation.userplaceholders <string list> Additional single-valued placeholders for validation. Supported types: int, float, bool. (optional, off by default)
-validation.usestreaming <boolean> Use the streaming through patches (slower but can process big dataset) (optional, off by default, default value is false)
-validation.source1 <group> Parameters for source #1 (validation)
-validation.source1.il <string list> Input image (or list to stack) for source #1 (validation) (mandatory)
-validation.source1.name <string> Name of the input placeholder or output tensor for source #1 (validation) (mandatory)
-validation.source2 <group> Parameters for source #2 (validation)
-validation.source2.il <string list> Input image (or list to stack) for source #2 (validation) (mandatory)
-validation.source2.name <string> Name of the input placeholder or output tensor for source #2 (validation) (mandatory)
-inxml <string> Load otb application from xml file (optional, off by default)
-progress <boolean> Report progress
-help <string list> Display long help (empty list), or help for given parameters keys
Use -help param1 [... paramN] to see detailed documentation of those parameters.
Examples:
otbcli_TensorflowModelTrain -source1.il spot6pms.tif -source1.placeholder x1 -source1.fovx 16 -source1.fovy 16 -source2.il labels.tif -source2.placeholder y1 -source2.fovx 1 -source2.fovy 1 -model.dir /tmp/my_saved_model/ -training.userplaceholders is_training=true dropout=0.2 -training.targetnodenames optimizer -model.saveto /tmp/my_saved_model_vars1
otbcli_TensorflowModelTrain -source1.il spot6pms.tif -source1.placeholder x1 -source1.patchsizex 16 -source1.patchsizey 16 -source2.il labels.tif -source2.placeholder y1 -source2.patchsizex 1 -source2.patchsizex 1 -model.dir /tmp/my_saved_model/ -training.userplaceholders is_training=true dropout=0.2 -training.targetnodes optimizer -model.saveto /tmp/my_saved_model_vars1
```
As you can note, there is `$OTB_TF_NSOURCES` + 1 sources for practical purpose: because we need at least 1 source for input data, and 1 source for the truth.
## Serve the model
......@@ -141,36 +141,34 @@ Like it was said before, the user is responsible of giving the *perceptive field
```
This is the (TensorflowModelServe) application, version 6.5.0
Multisource deep learning classifier using Tensorflow. Change the OTB_TF_NSOURCES environment variable to set the number of sources.
Parameters:
-source1 <group> Parameters for source #1
MISSING -source1.il <string list> Input image (or list to stack) for source #1 (mandatory)
MISSING -source1.fovx <int32> Field of view width for source #1 (mandatory)
MISSING -source1.fovy <int32> Field of view height for source #1 (mandatory)
MISSING -source1.placeholder <string> Name of the input placeholder for source #1 (mandatory)
-model <group> model parameters
MISSING -model.dir <string> Tensorflow model_save directory (mandatory)
-model.userplaceholders <string list> Additional single-valued placeholders. Supported types: int, float, bool. (optional, off by default)
-model.fullyconv <boolean> Fully convolutional (optional, off by default, default value is false)
-output <group> Output tensors parameters
-output.spcscale <float> The output spacing scale (mandatory, default value is 1)
MISSING -output.names <string list> Names of the output tensors (mandatory)
-output.foex <int32> The output field of expression (x) (mandatory, default value is 1)
-output.foey <int32> The output field of expression (y) (mandatory, default value is 1)
-finetuning <group> Fine tuning performance or consistency parameters
-finetuning.disabletiling <boolean> Disable tiling (optional, off by default, default value is false)
-finetuning.tilesize <int32> Tile width used to stream the filter output (mandatory, default value is 16)
MISSING -out <string> [pixel] output image [pixel=uint8/uint16/int16/uint32/int32/float/double] (default value is float) (mandatory)
-inxml <string> Load otb application from xml file (optional, off by default)
-progress <boolean> Report progress
-help <string list> Display long help (empty list), or help for given parameters keys
-source1 <group> Parameters for source #1
MISSING -source1.il <string list> Input image (or list to stack) for source #1 (mandatory)
MISSING -source1.rfieldx <int32> Input receptive field (width) for source #1 (mandatory)
MISSING -source1.rfieldy <int32> Input receptive field (height) for source #1 (mandatory)
MISSING -source1.placeholder <string> Name of the input placeholder for source #1 (mandatory)
-model <group> model parameters
MISSING -model.dir <string> Tensorflow model_save directory (mandatory)
-model.userplaceholders <string list> Additional single-valued placeholders. Supported types: int, float, bool. (optional, off by default)
-model.fullyconv <boolean> Fully convolutional (optional, off by default, default value is false)
-output <group> Output tensors parameters
-output.spcscale <float> The output spacing scale (mandatory, default value is 1)
MISSING -output.names <string list> Names of the output tensors (mandatory)
-output.efieldx <int32> The output expression field (width) (mandatory, default value is 1)
-output.efieldy <int32> The output expression field (height) (mandatory, default value is 1)
-optim <group> This group of parameters allows optimization of processing time
-optim.disabletiling <boolean> Disable tiling (optional, off by default, default value is false)
-optim.tilesize <int32> Tile width used to stream the filter output (mandatory, default value is 16)
MISSING -out <string> [pixel] output image [pixel=uint8/uint16/int16/uint32/int32/float/double/cint16/cint32/cfloat/cdouble] (default value is float) (mandatory)
-inxml <string> Load otb application from xml file (optional, off by default)
-progress <boolean> Report progress
-help <string list> Display long help (empty list), or help for given parameters keys
Use -help param1 [... paramN] to see detailed documentation of those parameters.
Examples:
otbcli_TensorflowModelServe -source1.il spot6pms.tif -source1.placeholder x1 -source1.fovx 16 -source1.fovy 16 -model.dir /tmp/my_saved_model/ -model.userplaceholders is_training=false dropout=0.0 -output.names out_predict1 out_proba1 -out "classif128tgt.tif?&streaming:type=tiled&streaming:sizemode=height&streaming:sizevalue=256" -finetuning.disabletiling
otbcli_TensorflowModelServe -source1.il spot6pms.tif -source1.placeholder x1 -source1.rfieldx 16 -source1.rfieldy 16 -model.dir /tmp/my_saved_model/ -model.userplaceholders is_training=false dropout=0.0 -output.names out_predict1 out_proba1 -out "classif128tgt.tif?&streaming:type=tiled&streaming:sizemode=height&streaming:sizevalue=256"
```
## Composite applications for classification
......@@ -179,14 +177,12 @@ This is possible thank to two new applications that uses the existing training/c
**TrainClassifierFromDeepFeatures**: is a composite application that wire the **TensorflowModelServe** application output into the existing official **TrainImagesClassifier** application.
```
This is the TrainClassifierFromDeepFeatures (TrainClassifierFromDeepFeatures) application, version 6.5.0
Train a classifier from deep net based features of an image and training vector data.
Parameters:
-source1 <group> Parameters for source 1
MISSING -source1.il <string list> Input image (or list to stack) for source #1 (mandatory)
MISSING -source1.fovx <int32> Field of view width for source #1 (mandatory)
MISSING -source1.fovy <int32> Field of view height for source #1 (mandatory)
MISSING -source1.rfieldx <int32> Input receptive field (width) for source #1 (mandatory)
MISSING -source1.rfieldy <int32> Input receptive field (height) for source #1 (mandatory)
MISSING -source1.placeholder <string> Name of the input placeholder for source #1 (mandatory)
-model <group> Deep net model parameters
MISSING -model.dir <string> Tensorflow model_save directory (mandatory)
......@@ -195,11 +191,11 @@ MISSING -model.dir <string> Tensorflow model_save dire
-output <group> Deep net outputs parameters
-output.spcscale <float> The output spacing scale (mandatory, default value is 1)
MISSING -output.names <string list> Names of the output tensors (mandatory)
-output.foex <int32> The output field of expression (x) (mandatory, default value is 1)
-output.foey <int32> The output field of expression (y) (mandatory, default value is 1)
-finetuning <group> Deep net fine tuning parameters
-finetuning.disabletiling <boolean> Disable tiling (optional, off by default, default value is false)
-finetuning.tilesize <int32> Tile width used to stream the filter output (mandatory, default value is 16)
-output.efieldx <int32> The output expression field (width) (mandatory, default value is 1)
-output.efieldy <int32> The output expression field (height) (mandatory, default value is 1)
-optim <group> This group of parameters allows optimization of processing time
-optim.disabletiling <boolean> Disable tiling (optional, off by default, default value is false)
-optim.tilesize <int32> Tile width used to stream the filter output (mandatory, default value is 16)
MISSING -vd <string list> Input vector data list (mandatory)
-valid <string list> Validation vector data list (optional, off by default)
MISSING -out <string> Output model (mandatory)
......@@ -209,7 +205,7 @@ MISSING -out <string> Output model (mandatory)
-sample.mv <int32> Maximum validation sample size per class (mandatory, default value is 1000)
-sample.bm <int32> Bound sample number by minimum (mandatory, default value is 1)
-sample.vtr <float> Training and validation sample ratio (mandatory, default value is 0.5)
-sample.vfn <string> Field containing the class integer label for supervision (mandatory, default value is )
-sample.vfn <string> Field containing the class integer label for supervision (mandatory, no default value)
-elev <group> Elevation management
-elev.dem <string> DEM directory (optional, off by default)
-elev.geoid <string> Geoid File (optional, off by default)
......@@ -273,14 +269,12 @@ Use -help param1 [... paramN] to see detailed documentation of those parameters.
**ImageClassifierFromDeepFeatures** same approach with the official **ImageClassifier**.
```
This is the ImageClassifierFromDeepFeatures (ImageClassifierFromDeepFeatures) application, version 6.5.0
Classify image using features from a deep net and an OTB machine learning classification model
Parameters:
-source1 <group> Parameters for source 1
MISSING -source1.il <string list> Input image (or list to stack) for source #1 (mandatory)
MISSING -source1.fovx <int32> Field of view width for source #1 (mandatory)
MISSING -source1.fovy <int32> Field of view height for source #1 (mandatory)
MISSING -source1.rfieldx <int32> Input receptive field (width) for source #1 (mandatory)
MISSING -source1.rfieldy <int32> Input receptive field (height) for source #1 (mandatory)
MISSING -source1.placeholder <string> Name of the input placeholder for source #1 (mandatory)
-deepmodel <group> Deep net model parameters
MISSING -deepmodel.dir <string> Tensorflow model_save directory (mandatory)
......@@ -289,16 +283,16 @@ MISSING -deepmodel.dir <string> Tensorflow model_save direc
-output <group> Deep net outputs parameters
-output.spcscale <float> The output spacing scale (mandatory, default value is 1)
MISSING -output.names <string list> Names of the output tensors (mandatory)
-output.foex <int32> The output field of expression (x) (mandatory, default value is 1)
-output.foey <int32> The output field of expression (y) (mandatory, default value is 1)
-finetuning <group> Deep net fine tuning parameters
-finetuning.disabletiling <boolean> Disable tiling (optional, off by default, default value is false)
-finetuning.tilesize <int32> Tile width used to stream the filter output (mandatory, default value is 16)
-output.efieldx <int32> The output expression field (width) (mandatory, default value is 1)
-output.efieldy <int32> The output expression field (height) (mandatory, default value is 1)
-optim <group> This group of parameters allows optimization of processing time
-optim.disabletiling <boolean> Disable tiling (optional, off by default, default value is false)
-optim.tilesize <int32> Tile width used to stream the filter output (mandatory, default value is 16)
MISSING -model <string> Model file (mandatory)
-imstat <string> Statistics file (optional, off by default)
-nodatalabel <int32> Label mask value (optional, off by default, default value is 0)
MISSING -out <string> [pixel] Output image [pixel=uint8/uint16/int16/uint32/int32/float/double] (default value is uint8) (mandatory)
-confmap <string> [pixel] Confidence map image [pixel=uint8/uint16/int16/uint32/int32/float/double] (default value is double) (optional, off by default)
MISSING -out <string> [pixel] Output image [pixel=uint8/uint16/int16/uint32/int32/float/double/cint16/cint32/cfloat/cdouble] (default value is uint8) (mandatory)
-confmap <string> [pixel] Confidence map image [pixel=uint8/uint16/int16/uint32/int32/float/double/cint16/cint32/cfloat/cdouble] (default value is double) (optional, off by default)
-ram <int32> Ram (optional, off by default, default value is 128)
-inxml <string> Load otb application from xml file (optional, off by default)
-progress <boolean> Report progress
......@@ -327,14 +321,14 @@ otbcli_PatchesExtraction -source1.il spot7.tif -source1.patchsizex 16 -source1.p
That's it. Now we have two images for patches and labels. If we wanna, we can split them to distinguish test/validation groups (with the **ExtractROI** application for instance). But here, we will just perform some fine tuning of our model, located in the `outmodel` directory. Our model is quite basic. It has two input placeholders, **x1** and **y1** respectively for input patches (with size 16x16) and input reference labels (with size 1x1). We named **prediction** the tensor that predict the labels and the optimizer that perform the stochastic gradient descent is an operator named **optimizer**. We perform the fine tuning and we export the new model variables in the `newvars` folder.
Let's use our **TensorflowModelTrain** application to perform the training of this existing model.
```
otbcli_TensorflowModelTrain -model.dir /path/to/oursavedmodel -training.targetnodesnames optimizer -training.source1.il samp_patches.tif -training.source1.fovx 16 -training.source1.fovy 16 -training.source1.placeholder x1 -training.source2.il samp_labels.tif -training.source2.fovx 1 -training.source2.fovy 1 -training.source2.placeholder y1 -model.saveto newvars
otbcli_TensorflowModelTrain -model.dir /path/to/oursavedmodel -training.targetnodesnames optimizer -training.source1.il samp_patches.tif -training.source1.patchsizex 16 -training.source1.patchsizey 16 -training.source1.placeholder x1 -training.source2.il samp_labels.tif -training.source2.patchsizex 1 -training.source2.patchsizey 1 -training.source2.placeholder y1 -model.saveto newvars
```
Note that we could also have performed validation in this step. In this case, the `validation.source2.placeholder` would be different than the `training.source2.placeholder`, and would be **prediction**. This way, the program know what is the target tensor to evaluate.
After this step, we decide to produce an entire map of forest over the whole Spot-7 image. First, we duplicate the model, and we replace its variable with the new ones that have been computed in the previous step.
Then, we use the **TensorflowModelServe** application to produce the **prediction** tensor output for the entire image.
```
otbcli_TensorflowModelServe -source1.il spot7.tif -source1.placeholder x1 -source1.fovx 16 -source1.fovy 16 -model.dir /tmp/my_new_model -output.names prediction -out map.tif uint8
otbcli_TensorflowModelServe -source1.il spot7.tif -source1.placeholder x1 -source1.rfieldx 16 -source1.rfieldy 16 -model.dir /tmp/my_new_model -output.names prediction -out map.tif uint8
```
# Contact
......
......@@ -92,9 +92,11 @@ private:
ShareParameter("deepmodel", "tfmodel.model",
"Deep net model parameters", "Deep net model parameters");
ShareParameter("output", "tfmodel.output",
"Deep net outputs parameters", "Deep net outputs parameters");
ShareParameter("finetuning", "tfmodel.finetuning",
"Deep net fine tuning parameters","Deep net fine tuning parameters");
"Deep net outputs parameters",
"Deep net outputs parameters");
ShareParameter("optim", "tfmodel.optim",
"This group of parameters allows optimization of processing time",
"This group of parameters allows optimization of processing time");
// Classify shared parameters
ShareParameter("model" , "classif.model" , "Model file" , "Model file" );
......
......@@ -106,14 +106,14 @@ public:
// Parameter group keys
ss_key_in << ss_key_group.str() << ".il";
ss_key_dims_x << ss_key_group.str() << ".fovx";
ss_key_dims_y << ss_key_group.str() << ".fovy";
ss_key_dims_x << ss_key_group.str() << ".rfieldx";
ss_key_dims_y << ss_key_group.str() << ".rfieldy";
ss_key_ph << ss_key_group.str() << ".placeholder";
// Parameter group descriptions
ss_desc_in << "Input image (or list to stack) for source #" << inputNumber;
ss_desc_dims_x << "Field of view width for source #" << inputNumber;
ss_desc_dims_y << "Field of view height for source #" << inputNumber;
ss_desc_dims_x << "Input receptive field (width) for source #" << inputNumber;
ss_desc_dims_y << "Input receptive field (height) for source #" << inputNumber;
ss_desc_ph << "Name of the input placeholder for source #" << inputNumber;
// Populate group
......@@ -182,22 +182,22 @@ public:
MandatoryOn ("output.names");
// Output Field of Expression
AddParameter(ParameterType_Int, "output.foex", "The output field of expression (x)");
SetMinimumParameterIntValue ("output.foex", 1);
SetDefaultParameterInt ("output.foex", 1);
MandatoryOn ("output.foex");
AddParameter(ParameterType_Int, "output.foey", "The output field of expression (y)");
SetMinimumParameterIntValue ("output.foey", 1);
SetDefaultParameterInt ("output.foey", 1);
MandatoryOn ("output.foey");
AddParameter(ParameterType_Int, "output.efieldx", "The output expression field (width)");
SetMinimumParameterIntValue ("output.efieldx", 1);
SetDefaultParameterInt ("output.efieldx", 1);
MandatoryOn ("output.efieldx");
AddParameter(ParameterType_Int, "output.efieldy", "The output expression field (height)");
SetMinimumParameterIntValue ("output.efieldy", 1);
SetDefaultParameterInt ("output.efieldy", 1);
MandatoryOn ("output.efieldy");
// Fine tuning
AddParameter(ParameterType_Group, "finetuning" , "Fine tuning performance or consistency parameters");
AddParameter(ParameterType_Bool, "finetuning.disabletiling", "Disable tiling");
MandatoryOff ("finetuning.disabletiling");
AddParameter(ParameterType_Int, "finetuning.tilesize", "Tile width used to stream the filter output");
SetMinimumParameterIntValue ("finetuning.tilesize", 1);
SetDefaultParameterInt ("finetuning.tilesize", 16);
AddParameter(ParameterType_Group, "optim" , "This group of parameters allows optimization of processing time");
AddParameter(ParameterType_Bool, "optim.disabletiling", "Disable tiling");
MandatoryOff ("optim.disabletiling");
AddParameter(ParameterType_Int, "optim.tilesize", "Tile width used to stream the filter output");
SetMinimumParameterIntValue ("optim.tilesize", 1);
SetDefaultParameterInt ("optim.tilesize", 16);
// Output image
AddParameter(ParameterType_OutputImage, "out", "output image");
......@@ -205,8 +205,8 @@ public:
// Example
SetDocExampleParameterValue("source1.il", "spot6pms.tif");
SetDocExampleParameterValue("source1.placeholder", "x1");
SetDocExampleParameterValue("source1.fovx", "16");
SetDocExampleParameterValue("source1.fovy", "16");
SetDocExampleParameterValue("source1.rfieldx", "16");
SetDocExampleParameterValue("source1.rfieldy", "16");
SetDocExampleParameterValue("model.dir", "/tmp/my_saved_model/");
SetDocExampleParameterValue("model.userplaceholders", "is_training=false dropout=0.0");
SetDocExampleParameterValue("output.names", "out_predict1 out_proba1");
......@@ -248,16 +248,16 @@ public:
m_TFFilter = TFModelFilterType::New();
m_TFFilter->SetGraph(m_SavedModel.meta_graph_def.graph_def());
m_TFFilter->SetSession(m_SavedModel.session.get());
m_TFFilter->SetOutputTensorsNames(GetParameterStringList("output.names"));
m_TFFilter->SetOutputTensors(GetParameterStringList("output.names"));
m_TFFilter->SetOutputSpacingScale(GetParameterFloat("output.spcscale"));
otbAppLogINFO("Output spacing ratio: " << m_TFFilter->GetOutputSpacingScale());
// Get user placeholders
TFModelFilterType::DictListType dict;
TFModelFilterType::StringList expressions = GetParameterStringList("model.userplaceholders");
TFModelFilterType::DictType dict;
for (auto& exp: expressions)
{
TFModelFilterType::DictType entry = tf::ExpressionToTensor(exp);
TFModelFilterType::DictElementType entry = tf::ExpressionToTensor(exp);
dict.push_back(entry);
otbAppLogINFO("Using placeholder " << entry.first << " with " << tf::PrintTensorInfos(entry.second));
......@@ -267,7 +267,7 @@ public:
// Input sources
for (auto& bundle: m_Bundles)
{
m_TFFilter->PushBackInputBundle(bundle.m_Placeholder, bundle.m_PatchSize, bundle.m_ImageSource.Get());
m_TFFilter->PushBackInputTensorBundle(bundle.m_Placeholder, bundle.m_PatchSize, bundle.m_ImageSource.Get());
}
// Fully convolutional mode on/off
......@@ -281,15 +281,15 @@ public:
FloatVectorImageType::SizeType foe;
foe[0] = GetParameterInt("output.foex");
foe[1] = GetParameterInt("output.foey");
m_TFFilter->SetOutputFOESize(foe);
m_TFFilter->SetOutputExpressionFields({foe});
otbAppLogINFO("Output field of expression: " << m_TFFilter->GetOutputFOESize());
otbAppLogINFO("Output field of expression: " << m_TFFilter->GetOutputExpressionFields()[0]);
// Streaming
if (GetParameterInt("finetuning.disabletiling")!=1)
if (GetParameterInt("optim.disabletiling")!=1)
{
// Get the tile size
const unsigned int tileSize = GetParameterInt("finetuning.tilesize");
const unsigned int tileSize = GetParameterInt("optim.tilesize");
otbAppLogINFO("Force tiling with squared tiles of " << tileSize)
// Update the TF filter to get the output image size
......
......@@ -53,7 +53,7 @@ public:
itkNewMacro(Self);
itkTypeMacro(TensorflowModelTrain, Application);
/** Typedefs for tensorflow */
/** Typedefs for TensorFlow */
typedef otb::TensorflowMultisourceModelTrain<FloatVectorImageType> TrainModelFilterType;
typedef otb::TensorflowMultisourceModelValidate<FloatVectorImageType> ValidateModelFilterType;
typedef otb::TensorflowSource<FloatVectorImageType> TFSource;
......@@ -75,8 +75,8 @@ public:
// Parameters keys
std::string m_KeyInForTrain; // Key of input image list (training)
std::string m_KeyInForValid; // Key of input image list (validation)
std::string m_KeyPHNameForTrain; // Key for placeholder name in the tensorflow model (training)
std::string m_KeyPHNameForValid; // Key for placeholder name in the tensorflow model (validation)
std::string m_KeyPHNameForTrain; // Key for placeholder name in the TensorFlow model (training)
std::string m_KeyPHNameForValid; // Key for placeholder name in the TensorFlow model (validation)
std::string m_KeyPszX; // Key for samples sizes X
std::string m_KeyPszY; // Key for samples sizes Y
};
......@@ -122,18 +122,19 @@ public:
// Parameter group keys
ss_key_tr_in << ss_key_tr_group.str() << ".il";
ss_key_val_in << ss_key_val_group.str() << ".il";
ss_key_dims_x << ss_key_tr_group.str() << ".fovx";
ss_key_dims_y << ss_key_tr_group.str() << ".fovy";
ss_key_dims_x << ss_key_tr_group.str() << ".patchsizex";
ss_key_dims_y << ss_key_tr_group.str() << ".patchsizey";
ss_key_tr_ph << ss_key_tr_group.str() << ".placeholder";
ss_key_val_ph << ss_key_val_group.str() << ".placeholder";
ss_key_val_ph << ss_key_val_group.str() << ".name";
// Parameter group descriptions
ss_desc_tr_in << "Input image (or list to stack) for source #" << inputNumber << " (training)";
ss_desc_val_in << "Input image (or list to stack) for source #" << inputNumber << " (validation)";
ss_desc_dims_x << "Field of view width for source #" << inputNumber;
ss_desc_dims_y << "Field of view height for source #" << inputNumber;
ss_desc_dims_x << "Patch size (x) for source #" << inputNumber;
ss_desc_dims_y << "Patch size (y) for source #" << inputNumber;
ss_desc_tr_ph << "Name of the input placeholder for source #" << inputNumber << " (training)";
ss_desc_val_ph << "Name of the input placeholder for source #" << inputNumber << " (validation)";
ss_desc_val_ph << "Name of the input placeholder "
"or output tensor for source #" << inputNumber << " (validation)";
// Populate group
AddParameter(ParameterType_Group, ss_key_tr_group.str(), ss_desc_tr_group.str());
......@@ -194,21 +195,25 @@ public:
AddParameter(ParameterType_StringList, "training.userplaceholders",
"Additional single-valued placeholders for training. Supported types: int, float, bool.");
MandatoryOff ("training.userplaceholders");
AddParameter(ParameterType_StringList, "training.targetnodesnames", "Names of the target nodes");
MandatoryOn ("training.targetnodesnames");
AddParameter(ParameterType_StringList, "training.outputtensorsnames", "Names of the output tensors to display");
MandatoryOff ("training.outputtensorsnames");
AddParameter(ParameterType_StringList, "training.targetnodes", "Names of the target nodes");
MandatoryOn ("training.targetnodes");
AddParameter(ParameterType_StringList, "training.outputtensors", "Names of the output tensors to display");
MandatoryOff ("training.outputtensors");
AddParameter(ParameterType_Bool, "training.usestreaming", "Use the streaming through patches (slower but can process big dataset)");
MandatoryOff ("training.usestreaming");
// Metrics
AddParameter(ParameterType_Group, "validation", "Validation parameters");
AddParameter(ParameterType_Group, "validation", "Validation parameters");
MandatoryOff ("validation");
AddParameter(ParameterType_Choice, "validation.mode", "Metrics to compute");
AddChoice ("validation.mode.none", "No validation step");
AddChoice ("validation.mode.class", "Classification metrics");
AddChoice ("validation.mode.rmse", "Root mean square error");
AddParameter(ParameterType_Choice, "validation.mode", "Metrics to compute");
AddChoice ("validation.mode.none", "No validation step");
AddChoice ("validation.mode.class", "Classification metrics");
AddChoice ("validation.mode.rmse", "Root mean square error");
AddParameter(ParameterType_StringList, "validation.userplaceholders",
"Additional single-valued placeholders for validation. Supported types: int, float, bool.");
MandatoryOff ("validation.userplaceholders");
AddParameter(ParameterType_Bool, "validation.usestreaming", "Use the streaming through patches (slower but can process big dataset)");
MandatoryOff ("validation.usestreaming");
// Input/output images
AddAnInputImage();
......@@ -220,15 +225,15 @@ public:
// Example
SetDocExampleParameterValue("source1.il", "spot6pms.tif");
SetDocExampleParameterValue("source1.placeholder", "x1");
SetDocExampleParameterValue("source1.fovx", "16");
SetDocExampleParameterValue("source1.fovy", "16");
SetDocExampleParameterValue("source1.patchsizex", "16");
SetDocExampleParameterValue("source1.patchsizey", "16");
SetDocExampleParameterValue("source2.il", "labels.tif");
SetDocExampleParameterValue("source2.placeholder", "y1");
SetDocExampleParameterValue("source2.fovx", "1");
SetDocExampleParameterValue("source2.fovy", "1");
SetDocExampleParameterValue("source2.patchsizex", "1");
SetDocExampleParameterValue("source2.patchsizex", "1");
SetDocExampleParameterValue("model.dir", "/tmp/my_saved_model/");
SetDocExampleParameterValue("training.userplaceholders", "is_training=true dropout=0.2");
SetDocExampleParameterValue("training.targetnodenames", "optimizer");
SetDocExampleParameterValue("training.targetnodes", "optimizer");
SetDocExampleParameterValue("model.saveto", "/tmp/my_saved_model_vars1");
}
......@@ -240,10 +245,10 @@ public:
// -Placeholders
// -PatchSize
// -ImageSource
// 2.Validation/Test
// 2.Learning/Validation
// -Placeholders (if input) or Tensor name (if target)
// -PatchSize (which is the same as for training)
// -ImageSource (depending if it's for test or validation)
// -ImageSource (depending if it's for learning or validation)
//
// TODO: a bit of refactoring. We could simply rely on m_Bundles
// if we can keep trace of indices of sources for
......@@ -262,12 +267,12 @@ public:
// Clear bundles
m_InputSourcesForTraining.clear();
m_InputSourcesForTest.clear();
m_InputSourcesForValidation.clear();
m_InputSourcesForEvaluationAgainstLearningData.clear();
m_InputSourcesForEvaluationAgainstValidationData.clear();
m_TargetTensorsNames.clear();
m_InputTargetsForValidation.clear();
m_InputTargetsForTest.clear();
m_InputTargetsForEvaluationAgainstValidationData.clear();
m_InputTargetsForEvaluationAgainstLearningData.clear();
// Prepare the bundles
......@@ -289,7 +294,7 @@ public:
m_InputPatchesSizeForTraining.push_back(patchSize);
otbAppLogINFO("New source:");
otbAppLogINFO("Field of view : "<< patchSize);
otbAppLogINFO("Patch size : "<< patchSize);
otbAppLogINFO("Placeholder (training) : "<< placeholderForTraining);
// Prepare validation sources
......@@ -314,8 +319,8 @@ public:
if (placeholderForValidation.compare(placeholderForTraining) == 0)
{
// Source
m_InputSourcesForValidation.push_back(bundle.tfSourceForValidation.Get());
m_InputSourcesForTest.push_back(bundle.tfSource.Get());
m_InputSourcesForEvaluationAgainstValidationData.push_back(bundle.tfSourceForValidation.Get());
m_InputSourcesForEvaluationAgainstLearningData.push_back(bundle.tfSource.Get());
// Placeholder
m_InputPlaceholdersForValidation.push_back(placeholderForValidation);
......@@ -330,8 +335,8 @@ public:
else
{
// Source
m_InputTargetsForValidation.push_back(bundle.tfSourceForValidation.Get());
m_InputTargetsForTest.push_back(bundle.tfSource.Get());
m_InputTargetsForEvaluationAgainstValidationData.push_back(bundle.tfSourceForValidation.Get());
m_InputTargetsForEvaluationAgainstLearningData.push_back(bundle.tfSource.Get());
// Placeholder
m_TargetTensorsNames.push_back(placeholderForValidation);
......@@ -350,13 +355,13 @@ public:
//
// Get user placeholders
//
TrainModelFilterType::DictListType GetUserPlaceholders(const std::string key)
TrainModelFilterType::DictType GetUserPlaceholders(const std::string key)
{
TrainModelFilterType::DictListType dict;
TrainModelFilterType::DictType dict;
TrainModelFilterType::StringList expressions = GetParameterStringList(key);
for (auto& exp: expressions)
{
TrainModelFilterType::DictType entry = tf::ExpressionToTensor(exp);
TrainModelFilterType::DictElementType entry = tf::ExpressionToTensor(exp);
dict.push_back(entry);
otbAppLogINFO("Using placeholder " << entry.first << " with " << tf::PrintTensorInfos(entry.second));
......@@ -414,16 +419,19 @@ public:
m_TrainModelFilter = TrainModelFilterType::New();
m_TrainModelFilter->SetGraph(m_SavedModel.meta_graph_def.graph_def());
m_TrainModelFilter->SetSession(m_SavedModel.session.get());
m_TrainModelFilter->SetOutputTensorsNames(GetParameterStringList("training.outputtensorsnames"));
m_TrainModelFilter->SetTargetNodesNames(GetParameterStringList("training.targetnodesnames"));
m_TrainModelFilter->SetOutputTensors(GetParameterStringList("training.outputtensors"));
m_TrainModelFilter->SetTargetNodesNames(GetParameterStringList("training.targetnodes"));
m_TrainModelFilter->SetBatchSize(GetParameterInt("training.batchsize"));
m_TrainModelFilter->SetUserPlaceholders(GetUserPlaceholders("training.userplaceholders"));
m_TrainModelFilter->SetUseStreaming(GetParameterInt("training.usestreaming"));
// Set input bundles
// Set inputs
for (unsigned int i = 0 ; i < m_InputSourcesForTraining.size() ; i++)
{
m_TrainModelFilter->PushBackInputBundle(m_InputPlaceholdersForTraining[i],
m_InputPatchesSizeForTraining[i], m_InputSourcesForTraining[i]);
m_TrainModelFilter->PushBackInputTensorBundle(
m_InputPlaceholdersForTraining[i],
m_InputPatchesSizeForTraining[i],
m_InputSourcesForTraining[i]);
}
// Train the model
......@@ -449,55 +457,55 @@ public:
m_ValidateModelFilter = ValidateModelFilterType::New();
m_ValidateModelFilter->SetGraph(m_SavedModel.meta_graph_def.graph_def());
m_ValidateModelFilter->SetSession(m_SavedModel.session.get());
m_ValidateModelFilter->SetOutputTensorsNames(m_TargetTensorsNames);
m_ValidateModelFilter->SetBatchSize(GetParameterInt("training.batchsize"));
m_ValidateModelFilter->SetUserPlaceholders(GetUserPlaceholders("validation.userplaceholders"));
// Evaluate the metrics against the learning data (test)
for (unsigned int i = 0 ; i < m_InputSourcesForTest.size() ; i++)
{
m_ValidateModelFilter->PushBackInputBundle(m_InputPlaceholdersForValidation[i],
m_InputPatchesSizeForValidation[i], m_InputSourcesForTest[i]);
}
for (unsigned int i = 0 ; i < m_TargetTensorsNames.size() ; i++)
// AS we use the learning data here, it's rational to use the same option as streaming during training
m_ValidateModelFilter->SetUseStreaming(GetParameterInt("training.usestreaming"));