DOC: update mkdoc and doc

doc/arch.md

 # Features
 
-List of main features
+`Scenes` is a small python library aiming to provide a unified access to common remote sensing products.
+Currently, supported sensors are:
 
-## Spatial and temporal indexation
-
-Perform a query in space (WGS84 bounding box) and time (optional) with an RTree indexation structure.
-```
-scenes = drs.get_spot67_scenes(root_dir)  # drs contains helpers to deal with Spot 6/7 geosud collections
-idx = indexation.Index(scenes)            # spatial/temporal index
-matches = idx.find(bbox_wgs84=bbox)       # Returns scenes list. Query can include dt_min/dt_max datetime
-for one_scene in matches:                 # Print matching scenes info
-  print(one_scene)
-```
+- Spot 6
+- Spot 7
+- Sentinel-2 (Theia, Level 2)
+- Sentinel-2 (Theia, Level 3)
 
 ## Imagery access
 
 ### Use cases
 
-XS image in TOA
-```
-toa = scene.get_imagery(reflectance="toa")  # imagery instance
-xs = toa.get_xs()                           # source instance
+Computing NDVI from XS image in TOA reflectance:
+
+``` py
+toa = sc.get_imagery(reflectance="toa")  # (1)
+xs = toa.get_xs()                        # (2)
 exp = "(im1b4-im1b1)/(im1b4+im1b1)"
-ndvi = pyotb.bandmath(exp=exp, il=[xs])     # pyotb.app using the source
+ndvi = pyotb.bandmath(exp=exp, il=[xs])  # (3)
 ndvi.write("ndvi.tif")
 ```
 
-Pansharpened XS image in TOA, masked with cloud mask, over a ROI:
-```
-toa = scene.get_imagery(reflectance="toa")  # imagery instance
-pxs = toa.get_pxs()                         # source instance (1)
-drilled = pxs.drilled()                     # source instance (2)
+1. `toa` is a `scenes.spot.Spot67Imagery` instance
+2. `xs` is a `scenes.spot.Spot67Source` instance
+3. `ndvi` is a `pyotb.app` that inputs `xs`
+
+The next example is a set of preprocessing operations on a Spot-6/7 XS image:
+
+1. TOA reflectance
+2. pansharpening
+3. replacing the pixels under the clouds masks with "0"
+4. clip the result over a reference raster
+
+``` py
+toa = scene.get_imagery(reflectance="toa")  # (1)
+pxs = toa.get_pxs()                         # (2)
+drilled = pxs.drilled()                     # (3)
 ref_img = "/tmp/S2A_2020...._FRE_10m.tif"
-subset = drilled.clip_over_img(ref_img)     # source instance (3)
+subset = drilled.clip_over_img(ref_img)     # (4)
 subset.write("subset.tif")
 ```
 
-Superimpose an image:
+1. `toa` is a `scenes.spot.Spot67Imagery` instance
+2. `pxs` is a `scenes.spot.Spot67Imagery` instance
+3. `drilled` is a `scenes.spot.Spot67Imagery` instance
+4. `subset` is a `scenes.spot.Spot67Imagery` instance
+
+Note that we could have changed the no-data value in the cloud masking:
+
+``` py
+drilled = pxs.drilled(nodata=-999)
 ```
-toa = scene.get_imagery(reflectance="toa")  # imagery instance
-superimposed = toa.resample_over(ref_img)   # source instance
+
+Superimpose an image over a reference image.
+In the example below, `ref_img` is another `scenes.core.Source` instance.
+
+``` py
+toa = scene.get_imagery(reflectance="toa")
+superimposed = toa.resample_over(ref_img)
 superimposed.write("superimposed.tif")
 ```
 
+## Spatial and temporal indexation
+
+Perform a query in space (WGS84 bounding box) and time (optional) with an indexation structure.
+``` py
+from scenes import spot, indexation
+scs = spot.get_spot67_scenes(root_dir)  # (1)
+idx = indexation.Index(scenes)          # (2)
+matches = idx.find(bbox_wgs84=bbox)     # (3)
+for sc in matches:
+  print(sc)
+```
+
+1. `scs` is a list of `scenes.core.Scene` instances
+2. `idx` is a `scenes.indexation.Index` instance, namely a spatio-temporal index
+3. `matches` is a list of `scenes.core.Scene` instances
+
+To perform searches in time:
+``` py
+matches1 = idx.find(bbox_wgs84=bbox, "01/02/2019", "01-12-2020")  # (1)
+matches2 = idx.find(bbox_wgs84=bbox, "01/02/2019")                # (2)
+matches3 = idx.find(bbox_wgs84=bbox, date_max="01/02/2019")       # (3)
+```
+
+1. A few date formats are supported: "dd/mm/yyyy", "dd-mm-yyyy", and "yyyy-mm-dd"
+2. Here we don't have upper bound
+3. You can also speficy only an upper bound, without lower bound (use the keywords `date_min` and `date_max`
+
+
 # Classes
 
 ## Scene class
@@ -50,7 +97,7 @@ superimposed.write("superimposed.tif")
 The scene class handles all the metadata and the imagery.
 
 
-```mermaid
+``` mermaid
 classDiagram
 
     Scene <|-- Spot67Scene
@@ -106,12 +153,12 @@ classDiagram
 
 ```
 
-## Imagery
+## Imagery class
 
 The imagery stores the images sources for a particular sensor.
 
 
-```mermaid
+``` mermaid
 classDiagram
 
     Imagery <|-- Spot67Imagery
@@ -150,11 +197,11 @@ classDiagram
 
 ```
 
-## Source
+## Source class
 
 The source stores the image pipeline that delivers the pixels.
 
-```mermaid
+``` mermaid
 classDiagram
 
     Source <|-- Spot67Source
@@ -194,3 +241,12 @@ classDiagram
     }
 
 ```
+
+## Implement a new sensor from scratch
+
+You can implement quickly a new sensor in `scenes`.
+
+- A new `MySensorScene` class, inheriting from `scenes.core.Scene`. This class must provide one or multiple methods to its imageries.
+- One or multiple `MySensorImagery1`, ..., `MySensorImageryN` classes, inheriting from `scenes.core.Imagery`. This class must provide one or multiple methods to access to its sources.
+- One or multiple `MySensorSource1`, ..., `MySensorSourceM` classes, inheriting from `scenes.core.Source`
+

doc/gen_ref_pages.py

@@ -6,22 +6,22 @@ import mkdocs_gen_files
 
 nav = mkdocs_gen_files.Nav()
 
-processed_paths = ["app", "scenes"]
+processed_paths = ["apps", "scenes"]
 
 for path in sorted(Path(".").rglob("*.py")):
-    print(f"path: {path}")
     module_path = path.relative_to(".").with_suffix("")
     doc_path = path.relative_to(".").with_suffix(".md")
     full_doc_path = Path("reference", doc_path)
+    print("\n")
+    print(f"path: {path}")
     print(f"module path: {module_path}")
     print(f"doc path:{doc_path}")
 
     parts = tuple(module_path.parts)
     print(f"parts: {parts}")
     if parts[0] not in processed_paths:
+        print("--> Ignored")
         continue
-        print("--------")
-
 
     if  parts[-1] == "__init__":
         if len(parts) <= 1:
@@ -32,8 +32,8 @@ for path in sorted(Path(".").rglob("*.py")):
     elif parts[-1] == "__main__":
         continue
 
-    print(f"doc path:{doc_path}")
-    print(f"parts: {parts}")
+    print(f"new doc path:{doc_path}")
+    print(f"new parts: {parts}")
     nav[parts] = doc_path.as_posix()
 
     with mkdocs_gen_files.open(full_doc_path, "w") as fd:

mkdocs.yml

@@ -3,9 +3,11 @@ theme:
   name: "material"
   icon:
     repo: material/github
-
+  features:
+    - content.code.annotate
+    
+    
 plugins:
-- mermaid2
 - search
 - gen-files:
     scripts:
@@ -16,10 +18,11 @@ plugins:
 - literate-nav:
     nav_file: SUMMARY.md
 - section-index
+- mermaid2
 
 nav:
-- Code Reference:  reference/ 
-- Accueil: arch.md
+- API:  reference/ 
+- Overview: arch.md
 
 # Customization
 extra:
@@ -29,6 +32,17 @@ extra:
     - icon: material/gitlab
       link: https://gitlab.irstea.fr/umr-tetis/scenes
 
+markdown_extensions:
+  - pymdownx.highlight:
+      anchor_linenums: true
+  - pymdownx.inlinehilite
+  - pymdownx.snippets
+  - pymdownx.superfences:
+      # make exceptions to highlighting of code:
+      custom_fences:
+        - name: mermaid
+          class: mermaid
+          format: !!python/name:mermaid2.fence_mermaid  
 # rest of the navigation..
 ::: 
 site_name: Scenes