Skip to content
GitLab
Commit 1c75c177 authored by Lozac'h Loic's avatar Lozac'h Loic
Browse files

version a diffuser

parent 56b951df
master cnes debug dev dev-sam
No related merge requests found
Hide whitespace changes
Inline Side-by-side
Showing
with 145 additions and 45 deletions
+145 -45
readme.md
+ 145
− 45
View file @ 1c75c177
......@@ -28,7 +28,7 @@ user@machine:~$ cd theiacesprod/
user@machine:~/theiacesprod$ git clone https://gitlab.irstea.fr/loic.lozach/theiacesprod.git .
user@machine:~/theiacesprod$ gedit config.ini
```
Dans le dossier projet, ouvrir le fichier config.ini et modifier les valeurs des variables DATAPATH, pour le dossier d'accès à vos données, ainsi que les logins et mots de passe de vos comptes sur les sites distributeurs des images SENTINEL:
Dans le dossier projet, ouvrir le fichier config.ini et modifier les valeurs des variables DATAPATH et DBPATH, pour le dossier d'accès à vos données et le dossier de sauvegarde de la base de données, ainsi que les logins et mots de passe de vos comptes sur les sites distributeurs des images SENTINEL:
- SCIHUB : [Copernicus Open Access Hub](https://scihub.copernicus.eu/dhus/#/home)
- PEPS : [Plateforme d'Exploitation des Produits Sentinel](https://peps.cnes.fr/rocket/#/home)
- THEIA : [Produits à valeur ajoutée et algorithmes pour les surfaces continentales](https://theia.cnes.fr/atdistrib/rocket/#/home)
......@@ -64,6 +64,11 @@ Pour lancer les containers en commande d'arrière plan:
user@machine:~/theiacesprod$ make start
```
Pour initialiser la base de données (opération à ne réaliser qu'une seule fois):
```
user@machine:~/theiacesprod$ make dbmigrate
```
Pour accéder au shell (bash) du container webtheia :
```
user@machine:~/theiacesprod$ make shell
......@@ -71,20 +76,157 @@ user@machine:~/theiacesprod$ make shell
# Usage
## Auto Production Service
```
otbuser@c50154f03c40:/work/python$ autoprodssm_service.py -h
usage: autoprodssm_service.py [-h] {ssmww,ssmfr} ...
Automatic production of THEIA CES products
positional arguments:
{ssmww,ssmfr} Choose production between SoilMoisture, FrozenPlot (not available) and Irrigation
(not available)
ssmww Automatic production of Soil Moisture maps with S2-L2A.
ssmfr Automatic production of Soil Moisture maps over France with S2-L3A and RPG
agricultural reference vector.
optional arguments:
-h, --help show this help message and exit
```
Auto product SoilMoisture Service world wide allows a process with Sentinel2 L2A against an agricultural plot reference in vector or raster format.
```
otbuser@c50154f03c40:/work/python$ autoprodssm_service.py ssmww -h
usage: autoprodssm_service.py ssmww [-h] -sd YYYY-MM-DD -ed YYYY-MM-DD -modeldir MODELDIR
[-skipdown {s1,s2,all,no}] [-overwrite] -agriref
{vector,raster} [-agri.vect.file FILE]
[-agri.vect.idfield STRING] [-agri.vect.lufield STRING]
[-agri.vect.luvalues LIST]
[-agri.vect.clipchoice {s2tile,none}]
[-agri.vect.clip.s2tile T##XXX] [-agri.rast.file FILE]
[-agri.rast.luvalues LIST] [-agri.rast.clipchoice {s2tile,aoi}]
[-agri.rast.clip.s2tile T##XXX] [-agri.rast.clip.aoi FILE]
optional arguments:
-h, --help show this help message and exit
-sd YYYY-MM-DD Starting date S1, ex. 2020-08-31
-ed YYYY-MM-DD Ending date S1, ex. 2020-09-31
-modeldir MODELDIR Directory to find both "SavedModel_wet" and "SavedModel_dry" tensorflow
model
-skipdown {s1,s2,all,no}
[Optional] Skip sentinel 1 or 2 or both downloading step (default no). This
allows user to filter cloudy Sentinel2 images by removing them from project
directory in order to obtain better quality on gapfilled NDVI. Also useful
if offline images can not be retrieve.
-overwrite [Optional] Overwrite already existing files (default False)
-agriref {vector,raster}
Choose between agricultural plot reference vector file or Copernicus
landcover map
-agri.vect.file FILE Agricultural plot reference vector file, required if agriref=vector
-agri.vect.idfield STRING
Labels field name , default ID_PARCEL (France RPG 2019 shapefile)
-agri.vect.lufield STRING
LandUse field name , default CODE_GROUP (France RPG 2019 shapefile)
-agri.vect.luvalues LIST
List of agricultural areas values in LandUse vector, default "all"(all
CODE_GROUP in France RPG 2019 shapefile), ex: "1 2 3 4"
-agri.vect.clipchoice {s2tile,none}
Choose between clippng with a sentinel2 tile (s2tile) or use whole vector
file (none), area size must be under half S2 tile (<5000km2).
-agri.vect.clip.s2tile T##XXX
Clip agricultural plot reference vector with Sentinel2 Tile, ex: T30UVU
-agri.rast.file FILE Copernicus landcover map at 20x20 degree tiles raster file and 100 meters
pixel resolution, required if agriref=raster
-agri.rast.luvalues LIST
List of agricultural areas values in Copernicus landuse map, default "30
40"(grassland and crops)
-agri.rast.clipchoice {s2tile,aoi}
Choose between clippng with a sentinel2 tile (s2tile) or a polygon vector
file (aoi), area size must be under half S2 tile (<5000km2).
-agri.rast.clip.s2tile T##XXX
Clip landcover map over Sentinel2 Tile, ex: T30UVU
-agri.rast.clip.aoi FILE
An user's AOI vector file, area size must be under half S2 tile (<5000km2).
```
Auto product SoilMoisture Service over France allows a process with Sentinel2 L3A against french agricultural plot reference (RPG).
```
otbuser@39a616d684e8:/data$ autoprodssm_service.py ssmfr -h
usage: autoprodssm_service.py ssmfr [-h] -sd YYYY-MM-DD -ed YYYY-MM-DD -modeldir MODELDIR
[-modelchoice MODELCHOICE] [-skipdown {s1,s2,all,no}]
[-addtodb] [-overwrite] -agri.vect.file FILE -agri.vect.pubyear
AAAA -agri.vect.s2tile T##XXX [-agri.vect.idfield STRING]
[-agri.vect.lufield STRING] [-agri.vect.luvalues LIST]
optional arguments:
-h, --help show this help message and exit
-sd YYYY-MM-DD Starting date S1, ex. 2020-08-31
-ed YYYY-MM-DD Ending date S1, ex. 2020-09-31
-modeldir MODELDIR Directory to find both "SavedModel_wet" and "SavedModel_dry" tensorflow
model
-modelchoice MODELCHOICE
[Optional] dry, wet or text file resulting from gpm pipeline, default dry
-skipdown {s1,s2,all,no}
[Optional] Skip sentinel 1 or 2 or both downloading step (default no). This
allows user to filter cloudy Sentinel2 images by removing them from project
directory in order to obtain better quality on gapfilled NDVI. Also useful
if offline images can not be retrieve.
-addtodb [Optional] Add soilmoisture results in database (default False)
-overwrite [Optional] Overwrite already existing files (default False)
-agri.vect.file FILE Reseau Parcellaire Graphique (RPG) vector file
-agri.vect.pubyear AAAA
Year of publication of RPG
-agri.vect.s2tile T##XXX
Clip agricultural plot reference vector with Sentinel2 Tile, ex: T30UVU
-agri.vect.idfield STRING
Labels field name , default ID_PARCEL (France RPG 2019 shapefile)
-agri.vect.lufield STRING
LandUse field name , default CODE_GROUP (France RPG 2019 shapefile)
-agri.vect.luvalues LIST
List of agricultural areas values in LandUse vector, default "all"(all
CODE_GROUP in France RPG 2019 shapefile), ex: "1 2 3 4"
```
## Download Service
```
otbuser@50edfb299028:/data$ download_service.py eodag -h
otbuser@50edfb299028:/data$ download_service.py -h
usage: download_service.py [-h] {eodag,cds} ...
Download services
positional arguments:
{eodag,cds} Choose server
eodag Download S1 through Eodag API
cds Download ERA5 temperature image file according to S1 dates
```
Eodag is an API to wrap multiple imagery provider.
```
otbuser@39a616d684e8:/data$ download_service.py eodag -h
usage: download_service.py eodag [-h] -downdir DOWNDIR [-prod {0,1,2}] -tile TILE -sd SD -ed ED
optional arguments:
-h, --help show this help message and exit
-downdir DOWNDIR Download directory
-prod {0,1,2} Product type to download, choices 0=S1-GRD,1=S2-L2A,2=S2-L3A
-prod {0,1,2} Product type to download, choices 0=S1-GRD,1=S2-L2A,2=S2-L3A (FRANCE)
-tile TILE S2 Tile name, ex. T30UVU
-sd SD Starting date S1, ex. 2020-08-31
-ed ED Ending date S1, ex. 2020-09-31
```
CDS (Climate Data Store) is used for Freeze Detection processes
```
otbuser@39a616d684e8:/data$ download_service.py cds -h
usage: download_service.py cds [-h] -s1orthodir S1ORTHODIR [-gap {3m,6d}] -outdir OUTDIR
optional arguments:
-h, --help show this help message and exit
-s1orthodir S1ORTHODIR
Orthorectified Sentinel1 directory
-gap {3m,6d} [Optional] Query ERA5-Land reanalysis collection with 3 month gap (default
'3m') OR ERA5 basic reanalysis collection with 6 days gap ('6d')
-outdir OUTDIR Output directory
```
## Calibration Service
......@@ -162,49 +304,7 @@ optional arguments:
```
## Auto Production Service
```
otbuser@c50154f03c40:/work/python$ autoprod_service.py -h
usage: autoprod_service.py [-h] {ssm,freeze} ...
Automatic production of THEIA CES products
positional arguments:
{ssm,freeze} Choose pipeline
ssm Automatic production of Soil Moisture from S2 tile name, Agricultural RPG shapefile and time period.
freeze Automatic Freeze Detection from S2 tile name, Agricultural RPG shapefile and time period.
optional arguments:
-h, --help show this help message and exit
```
```
otbuser@c50154f03c40:/work/python$ autoprod_service.py ssm -h
usage: autoprod_service.py ssm [-h] -s2tile S2TILE -sd SD -ed ED -modeldir MODELDIR [-modelchoice MODELCHOICE] -rgpshp RGPSHP [-labelsfield LABELSFIELD] [-landusefield LANDUSEFIELD]
[-agrivalues AGRIVALUES] [-slopevalue SLOPEVALUE] [-sarmode {vv,vh}] [-overwrite]
optional arguments:
-h, --help show this help message and exit
-s2tile S2TILE Sentinel2 tile name, ex: T30UVU
-sd SD Starting date S1, ex. 2020-08-31
-ed ED Ending date S1, ex. 2020-09-31
-modeldir MODELDIR Directory to find both "SavedModel_wet" and "SavedModel_dry" tensorflow model
-modelchoice MODELCHOICE
[Optional] dry, wet or text file resulting from gpm pipeline, default dry
-rgpshp RGPSHP RGP (Reseau Graphique Parcellaire) shapefile or geopackage
-labelsfield LABELSFIELD
[Optional] Labels field name , default ID_PARCEL (France RPG 2019 shapefile)
-landusefield LANDUSEFIELD
[Optional] LandUse field name , default CODE_GROUP (France RPG 2019 shapefile)
-agrivalues AGRIVALUES
[Optional] List of agricultural areas values in LandUse raster, default "all"(all CODE_GROUP in France RPG 2019 shapefile), ex: "1 2 3 4"
-slopevalue SLOPEVALUE
[Optional] Max value of slope to use to filter SarVV, default 20
-sarmode {vv,vh} [Optional] Choose between VV or VH mode for input SAR image, default vv
-overwrite [Optional] Overwrite already existing files (default False)
```
## License
Please see the license for legal issues on the use of the software (GNU Affero General Public License v3.0).
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment