1. 13 Jan, 2023 2 commits
  2. 11 Jan, 2023 1 commit
  3. 04 Jan, 2023 1 commit
  4. 27 Dec, 2022 2 commits
  5. 21 Dec, 2022 1 commit
  6. 01 Dec, 2022 3 commits
  7. 20 Oct, 2022 1 commit
    • Thibault Hallouin's avatar
      pave the way for summary statistics on bootstrap samples · d37e69d2
      Thibault Hallouin authored
      Ultimately, the objective is for the user to be able to get the raw
      sampled metric values, or the mean and standard deviation of the sampled
      metric values, or a series of quantiles of the sampled metric values.
      There are still problems with the standard deviation on rtensor, and
      the computation of the quantiles does not work on n-dim expressions yet.
      So the second and third options are not possible yet, so only the raw
      values can be returned. Nonetheless, the machinery and the choice of
      where to introduce the summary functionality could be implemented,
      which is the purpose of this commit. A new parameter of the bootstrap
      experiment called "summary" is added: it can be given a value of 0 (to
      get the raw values). In the future, it would also take a value of 1 for
      mean+std, and 2 for quantiles.
      d37e69d2
  8. 06 Oct, 2022 1 commit
    • Thibault Hallouin's avatar
      implement bootstrapping method for metric uncertainty estimation · 16ce8f4e
      Thibault Hallouin authored
      The bootstrapping method is based on a non-overlapping block sampling
      with replacement, where the blocks are years of data. The number of
      samples and the sample length (i.e the number of year blocks) are both
      customisable.
      
      The method is accessible both for deterministic and probabilistic
      evaluation where a new axis is added. For now, the metrics for all the
      samples are returned, but in the future, some summary statistics would
      be implemented (e.g. quantiles or mean/standard deviation).
      
      /!\ For determinist evaluation, the n-dimensional functionality became
          untenable such that the number of dimensions was fixed and
          restricted to 2D tensors.
      
      New unit tests are included to test both the bootstrapping generator
      and the numerical results obtained with the bootstrapping turned on.
      16ce8f4e
  9. 30 Sep, 2022 1 commit
  10. 15 Sep, 2022 1 commit
  11. 13 Sep, 2022 1 commit
    • Thibault Hallouin's avatar
      allow masking conditions to be specified on predictions · f31664dd
      Thibault Hallouin authored
      An earlier implementation of the masking conditions assumed that the
      conditions on streamflow would only be on the observations, but this is
      not always the case. For example, reliability scores cannot be done on
      the observed streamflow and need to be performed on the predicted
      streamflow. So this is now possible as the condition syntax is changed
      and now *q_obs*/*q_prd_median*/*q_prd_mean* in place of *q*.
      f31664dd
  12. 12 Sep, 2022 1 commit
  13. 02 Sep, 2022 1 commit
  14. 31 Aug, 2022 1 commit
    • Thibault Hallouin's avatar
      implement functionality to generate temporal masks from conditions · b13d2f21
      Thibault Hallouin authored
      This functionality is inherited from `evalhyd-cli`. It allows the user
      to provide conditions as strings to specify how to generate temporal
      subsets. Conditions can be based on observed streamflow values (e.g.
      q>800, q<=120) or on time indices (e.g. to select particular events).
      
      This functionality is made available both for determinist and
      probabilist evaluation, unlike in  `evalhyd-cli` where it was only
      available for probabilist evaluation.
      
      This is documented in the docstrings, and new unit tests are written.
      b13d2f21
  15. 19 Aug, 2022 2 commits
    • Thibault Hallouin's avatar
      deal with missing data flagged as NaN in observations/predictions · 397501ad
      Thibault Hallouin authored
      The general approach is to "eliminate" the time steps where observations
      or predictions are missing as early as possible in the algorithm. The
      best approach seemed to update the user-provided temporal masks to
      also mask those time steps with missing data.
      
      An alternative approach would have been to create a view on the
      observations and predictions, e.g. using something like
      `xt::view(obs, ..., xt::drop(...))`, but this produces a non-contiguous
      view which cannot be sorted with `xt::sort` later to determine the
      quantiles.
      
      This is documented in `evalp` docstring and new unit tests are added.
      397501ad
    • Thibault Hallouin's avatar
      fix typo in unit test on masking · 41a30d84
      Thibault Hallouin authored
      resulting in only checking the first metric (i.e. BS) repeatedly
      41a30d84
  16. 10 Aug, 2022 1 commit
  17. 08 Aug, 2022 2 commits
  18. 11 Jul, 2022 1 commit
  19. 30 Jun, 2022 2 commits
  20. 29 Jun, 2022 1 commit
    • Thibault Hallouin's avatar
      add dimensions for sites/lead times to probabilistic evaluator · 295b3208
      Thibault Hallouin authored
      Internally, rather than using the multi-dimensional character of
      tensors to compute all sites and all lead times at once, loops are
      performed for each site and each lead time, in turn, in order to
      minimise memory imprint. Although at the moment, the input tensors are
      expected to feature the sites and lead times dimensions. If memory is
      an issue, the user can still send smaller tensors with size 1 for those
      dimensions and recompose multi-sites/multi-lead times output arrays
      externally.
      295b3208
  21. 15 Jun, 2022 1 commit
  22. 13 Jun, 2022 1 commit
    • Thibault Hallouin's avatar
      return squeezed xarray · 91aec5e1
      Thibault Hallouin authored
      Since the metrics are typically summary statistics, their size is not
      very big, so that using xtensor instead of xarray as a data structure is
      not as critical as for input data. In turn, using xarray allows for
      metrics of different sizes to be returned without unnecessary size 1
      dimensions (e.g. when only one threshold is given, or when no temporal
      masking is performed). So all output metrics are now returned in their
      "natural" shape (e.g. 1D for mono-component metrics, 2D for
      multi-component metrics), plus any additional dimension linked to
      multi-thresholds, multi-masking, etc.
      91aec5e1
  23. 10 Jun, 2022 1 commit
  24. 02 Jun, 2022 3 commits
  25. 25 May, 2022 1 commit
  26. 24 May, 2022 1 commit
  27. 17 May, 2022 1 commit
  28. 16 May, 2022 1 commit