otbtf issueshttps://gitlab.irstea.fr/remi.cresson/otbtf/-/issues2024-01-09T16:26:40+01:00https://gitlab.irstea.fr/remi.cresson/otbtf/-/issues/49Improve GDAL formats support2024-01-09T16:26:40+01:00Cresson RemiImprove GDAL formats support- zarr blosc compression
- geoparquet apache arrow
- libwebp- zarr blosc compression
- geoparquet apache arrow
- libwebphttps://gitlab.irstea.fr/remi.cresson/otbtf/-/issues/48Bug with non overlapping multisources2023-10-06T12:41:07+02:00Cresson RemiBug with non overlapping multisourcespan="https://minio-api-dinamis.apps.okd.crocc.meso.umontpellier.fr/catalog/spot67/SPOT7_MS_202207121015304_SPOT7_P_202207121015304/COG_SPOT7_P_202207121015304_ORT_SPOT7_20220713_1331561rgekx3ul1x1g_1.tif"
xs="https://minio-api-dinamis.a...pan="https://minio-api-dinamis.apps.okd.crocc.meso.umontpellier.fr/catalog/spot67/SPOT7_MS_202207121015304_SPOT7_P_202207121015304/COG_SPOT7_P_202207121015304_ORT_SPOT7_20220713_1331561rgekx3ul1x1g_1.tif"
xs="https://minio-api-dinamis.apps.okd.crocc.meso.umontpellier.fr/catalog/spot67/SPOT7_MS_202207121015304_SPOT7_P_202207121015304/COG_SPOT7_MS_202207121015304_ORT_SPOT7_20220713_1331511awwhqclkre47_1.tif"
- rf 64
- rf 16
- ef 32
- spcsc 1https://gitlab.irstea.fr/remi.cresson/otbtf/-/issues/47Release 4.2.02023-09-12T22:12:48+02:00Cresson RemiRelease 4.2.0- [ ] release notes
- [ ] doc
- [ ] setup.py- [ ] release notes
- [ ] doc
- [ ] setup.pyhttps://gitlab.irstea.fr/remi.cresson/otbtf/-/issues/46Add decloud in testing2023-08-26T21:34:11+02:00Cresson RemiAdd decloud in testinghttps://gitlab.irstea.fr/remi.cresson/otbtf/-/issues/45Update docker image2023-08-22T20:11:23+02:00Cresson RemiUpdate docker imageIt looks like today's "CRC book" test failed, even we don't have changed OTB/TF versions.
To be investigated, maybe some change in python packages?It looks like today's "CRC book" test failed, even we don't have changed OTB/TF versions.
To be investigated, maybe some change in python packages?https://gitlab.irstea.fr/remi.cresson/otbtf/-/issues/44Add helpers for common preprocessing functions2023-08-22T20:11:23+02:00Cresson RemiAdd helpers for common preprocessing functionse.g.
```python
tf.one_hot(
tf.squeeze(tf.cast(sample["labels_patches"], tf.int32), axis=-1),
depth=nb_cls
)
```
```python
class DilatedMask(keras.layers.Layer):
def __init__(self, nodata_value, dilatation_radius, name=Non...e.g.
```python
tf.one_hot(
tf.squeeze(tf.cast(sample["labels_patches"], tf.int32), axis=-1),
depth=nb_cls
)
```
```python
class DilatedMask(keras.layers.Layer):
def __init__(self, nodata_value, dilatation_radius, name=None):
self.nodata_value = nodata_value
self.dilatation_radius = dilatation_radius
super().__init__(name=name)
def call(self, inp):
"""
:param inp: input layer
"""
# Compute a binary mask from the input
nodata_mask = tf.cast(tf.math.equal(inp, self.nodata_value), tf.uint8)
dilatation_size = 1 + 2 * self.dilatation_radius
# Create a morphological kernel suitable for binary dilatation, cf https://stackoverflow.com/q/54686895/13711499
kernel = tf.zeros((dilatation_size, dilatation_size, 1), dtype=tf.uint8)
return tf.cast(tf.nn.dilation2d(input=nodata_mask, filters=kernel, strides=[1, 1, 1, 1], padding="SAME",
data_format="NHWC", dilations=[1, 1, 1, 1], name="dilatation_tf"), tf.uint8)
class ApplyMask(keras.layers.Layer):
def __init__(self, out_nodata, name=None):
super().__init__(name=name)
self.out_nodata = out_nodata
def call(self, inputs):
"""
:param inputs: [mask, input]. Mask is a binary mask, where 1 indicate the values to be masked on the input.
"""
mask, inp = inputs
return tf.where(mask == 1, float(self.out_nodata), inp)
class ScalarsTile(keras.layers.Layer):
"""
Duplicate some scalars in an whole array.
Simple example with only one scalar = 0.152: output [[0.152, 0.152, 0.152],
[0.152, 0.152, 0.152],
[0.152, 0.152, 0.152]]
"""
def __init__(self, name=None):
super().__init__(name=name)
def call(self, inputs):
"""
:param inputs: [reference, scalar inputs]. Reference is the tensor whose shape has to be matched
"""
ref, scalar_inputs = inputs
inp = tf.stack(scalar_inputs, axis=-1)
inp = tf.expand_dims(tf.expand_dims(inp, axis=1), axis=1)
return tf.tile(inp, [1, tf.shape(ref)[1], tf.shape(ref)[2], 1])
class Argmax(keras.layers.Layer):
"""
Compute the argmax of a tensor. For example, for a vector A=[0.1, 0.3, 0.6], the output is 2 (A[2] is the max)
Useful to transform a probability multibands map into a categorical map
"""
def __init__(self, name=None):
super().__init__(name=name)
def call(self, inputs):
return tf.expand_dims(tf.math.argmax(inputs, axis=-1), axis=-1)
class Max(keras.layers.Layer):
"""
Compute the max of a tensor. For example, for a vector [0.1, 0.3, 0.6], the output is 0.6
Useful to transform a probability multibands map into a "confidence" map
"""
def __init__(self, name=None):
super().__init__(name=name)
def call(self, inputs):
return tf.expand_dims(tf.math.reduce_max(inputs, axis=-1), axis=-1)
```https://gitlab.irstea.fr/remi.cresson/otbtf/-/issues/43Ultra-lightweight OTBTF image with otbapps only2023-06-09T17:25:10+02:00Cresson RemiUltra-lightweight OTBTF image with otbapps onlyCould be great to have an ultralight image optimized for inference jobs in production environments.
- only OTB+TF binaries
- no TF python package, and none of its dependency
- in short, keep only libtensorflow*.so and throw everything e...Could be great to have an ultralight image optimized for inference jobs in production environments.
- only OTB+TF binaries
- no TF python package, and none of its dependency
- in short, keep only libtensorflow*.so and throw everything else. Keep the OTB stuff.https://gitlab.irstea.fr/remi.cresson/otbtf/-/issues/42TensorRT support2023-05-19T09:22:27+02:00Cresson RemiTensorRT support# Notes
à ajouter au docker file pour compiler avec le support de TensorRT
```
RUN TRTDIR=/opt/otbtf/local/lib/python3.10/dist-packages/tensorflow/include/third_party/tensorrt/ && \
mkdir $TRTDIR && \
ln -s $(find /usr/ -iname "...# Notes
à ajouter au docker file pour compiler avec le support de TensorRT
```
RUN TRTDIR=/opt/otbtf/local/lib/python3.10/dist-packages/tensorflow/include/third_party/tensorrt/ && \
mkdir $TRTDIR && \
ln -s $(find /usr/ -iname "nvinfer.h") $TRTDIR
```https://gitlab.irstea.fr/remi.cresson/otbtf/-/issues/41Improve documentation2023-04-24T16:15:07+02:00Cresson RemiImprove documentation- [ ] pip install otbtf (standalone)- [ ] pip install otbtf (standalone)https://gitlab.irstea.fr/remi.cresson/otbtf/-/issues/40TensorflowModelServe nodata value2023-05-23T22:39:28+02:00Cresson RemiTensorflowModelServe nodata value# Possible improvement
- if no-data value is not provided: do as usual
- if no-data value is provided: test if all pixels are nodata in current chunk. If yes, skip inference.# Possible improvement
- if no-data value is not provided: do as usual
- if no-data value is provided: test if all pixels are nodata in current chunk. If yes, skip inference.https://gitlab.irstea.fr/remi.cresson/otbtf/-/issues/39Documentation: readthedocs?2023-04-04T12:25:13+02:00Cresson RemiDocumentation: readthedocs?https://gitlab.irstea.fr/remi.cresson/otbtf/-/issues/38Docker: push "latest" tag2022-10-04T14:35:46+02:00Cresson RemiDocker: push "latest" taghttps://gitlab.irstea.fr/remi.cresson/otbtf/-/issues/37Release 3.3.22022-09-22T13:08:23+02:00Cresson RemiRelease 3.3.2Cresson RemiCresson Remihttps://gitlab.irstea.fr/remi.cresson/otbtf/-/issues/36Move to OTB 8.1.02022-09-21T17:35:21+02:00Cresson RemiMove to OTB 8.1.0https://gitlab.irstea.fr/remi.cresson/otbtf/-/issues/34Upgrade pip in the docker image2022-09-19T20:51:43+02:00Cresson RemiUpgrade pip in the docker imageadd a `pip install pip --upgrade`add a `pip install pip --upgrade`https://gitlab.irstea.fr/remi.cresson/otbtf/-/issues/33Documentation: how to build and train a model with keras2023-04-04T12:25:13+02:00Cresson RemiDocumentation: how to build and train a model with kerasUse the FOSS4G-2022 slidesUse the FOSS4G-2022 slideshttps://gitlab.irstea.fr/remi.cresson/otbtf/-/issues/32Keep metadata in PatchesExtraction2022-09-12T09:41:47+02:00Cresson RemiKeep metadata in PatchesExtractionWe have to keep the input images metadatas in the output patches images.We have to keep the input images metadatas in the output patches images.https://gitlab.irstea.fr/remi.cresson/otbtf/-/issues/31GDAL not built with GEOS support2022-09-13T09:12:01+02:00Cresson RemiGDAL not built with GEOS supportsee https://stackoverflow.com/questions/70666700/compile-gdal-with-geossee https://stackoverflow.com/questions/70666700/compile-gdal-with-geosCresson RemiCresson Remihttps://gitlab.irstea.fr/remi.cresson/otbtf/-/issues/30Improve plot script2022-09-20T10:27:38+02:00Cresson RemiImprove plot scriptImprove the script (see deepmapper)Improve the script (see deepmapper)https://gitlab.irstea.fr/remi.cresson/otbtf/-/issues/29Add a Model class in OTBTF2022-07-28T07:35:49+02:00Narcon NicolasAdd a Model class in OTBTFOTBTF python now contains a Dataset and TFRecords classes, the next step could be to create a Model class (inside `model.py`?)OTBTF python now contains a Dataset and TFRecords classes, the next step could be to create a Model class (inside `model.py`?)Narcon NicolasNarcon Nicolas