GitLab

Problem

We face a (FATAL) TensorflowModelServe: Caught std::exception during application execution: std::bad_alloc when the output tensors of a model are not in the right shape (i.e. the last component is not the number of channels). While this is nominal that the error occurs, we should be able to explain the user why, instead of having this incomprehensible std::bad_alloc message 😉

How to reproduce

Model:

import tensorflow as tf

# Input
x = tf.keras.Input(shape=[None, None, None], name="x")  # [1, h, w, N]

# Compute norm on the last axis
y = tf.norm(x, axis=-1)

# Create model
model = tf.keras.Model(inputs={"x": x}, outputs={"y": y}, name="my_model")
model.save("my_model")

Inference:

We run the model in fully convolutional, meaning that the input image is processed in blocks of [1, h, w, N].

otbcli_TensorflowModelServe \
-source1.il sr4rs_data/input/SENTINEL2B_20200929-104857-489_L2A_T31TEJ_C_V2-2_FRE_10m.tif \
-model.dir my_model/ -out norm.tif -model.fullyconv on

What to tell to the user instead?

Explain him how to arrange output tensor dimensions. It is already in the doc but the point is to have some explicit warning/error message/suggestion message

Improve OTBTF to work with dimension-3 tensors

We can assume that 3-dimensional tensors are always [batch, h, x]. I don't see why a model would produce otherwise.