Commit 50effeed authored by Dorchies David's avatar Dorchies David
Browse files

Merge branch '47-automatically-update-website-from-package-repository' into 'dev'

Resolve "Automatically update website from package repository"

Closes #47

See merge request !15
parents 6d93d8d9 10d3d0cc
Pipeline #23455 passed with stages
in 20 minutes and 2 seconds
stages:
- checks
- website
default:
tags: [docker]
......@@ -23,12 +24,11 @@ before_script:
- mkdir -p $R_LIBS_USER $BUILD_LOGS_DIR
- echo "R_LIBS='$R_LIBS_USER'" > .Renviron
- R -e 'devtools::install_deps(dep = T)'
- R -e 'devtools::install_dev_deps()'
.check:
stage: checks
script:
- if [[ $NOT_CRAN == "false" ]]; then sudo apt-get install -y qpdf; fi
- if [[ $NOT_CRAN == "false" ]]; then sudo apt-get update && sudo apt-get install -y qpdf; fi
- R -e 'devtools::check(check_dir = Sys.getenv("CHECK_DIR"), cran = !as.logical(Sys.getenv("NOT_CRAN")), env_vars = c(NOT_CRAN = Sys.getenv("NOT_CRAN")))'
- R -e 'if (length(devtools::check_failures(path = Sys.getenv("BUILD_LOGS_DIR"), note = FALSE)) > 0) stop()'
......@@ -41,3 +41,15 @@ check_as_cran:
variables:
NOT_CRAN: "false"
extends: .check
website:
stage: website
# only:
# refs:
# - master
# - tags
script:
- R -e 'remotes::install_gitlab("in-wop/seinebasin", host = "gitlab.irstea.fr")'
- R -e 'pkgdown::build_site()'
- sudo apt-get update && sudo apt-get install -y sshpass rsync
- sshpass -p "${OVH_PASS}" rsync -a -e "ssh -o StrictHostKeyChecking=no" docs/ ${OVH_LOGIN}@${OVH_SFTP}:/home/${OVH_LOGIN}/airgriwrm/
......@@ -5,25 +5,29 @@ vignette: >
%\VignetteIndexEntry{Seine_01: Structuration of a semi-distributive GR4J model network}
%\VignetteEngine{knitr::rmarkdown}
%\VignetteEncoding{UTF-8}
bibliography: seinebasin.bib
---
```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
```
```{r}
```{r library}
library(airGRiwrm)
```
The example below is inspired by the semi-distributed model developed in the ClimAware project [@theobaldClimAwareImpactsClimate2014a].
## Semi-distributive network description
The model is distributed according to the gauging stations described in @dorchiesClimateChangeImpacts2014.
List of nodes
```{r}
```{r seine_nodes}
seine_nodes <- read.table(
file = system.file("seine_data", "network_gauging_stations.txt", package = "seinebasin"),
sep = ";", header = TRUE, fileEncoding = "UTF-8", quote = "\""
sep = ";", header = TRUE, fileEncoding = "UTF-8", quote = "\"", stringsAsFactors = FALSE
)
seine_nodes
```
......@@ -38,7 +42,7 @@ Create the GRiwrm object which lists the nodes and describes the network diagram
`GRiwrm` function helps to rename the columns of the dataframe and assign the variable classes.
```{r}
```{r griwrm}
seine_nodes$id_aval[seine_nodes$id_aval == ""] <- NA
seine_nodes$distance_aval <- as.double(seine_nodes$distance_aval) / 1000
seine_nodes$model <- "RunModel_GR4J"
......@@ -49,7 +53,7 @@ griwrm
The diagram of the network structure is represented below with in blue the upstream nodes with a GR4J model and in green the intermediate nodes with an SD (GR4J + LAG) model.
```{r fig.height = 7, fig.width = 8}
```{r DiagramGRiwrm, fig.height = 7, fig.width = 8}
DiagramGRiwrm(griwrm)
```
......@@ -57,48 +61,15 @@ DiagramGRiwrm(griwrm)
## Observation time series
Loading hydrometeorological data on the Seine river basin from the ClimAware project:
```{r, warning=FALSE, message=FALSE}
Precip <- NULL
PotEvap <- NULL
Qobs <- NULL
MergeTS <- function(dfOld, id, dfNew) {
names(dfNew) <- c("DatesR", id) # Renaming columns of the new input into date and sub-basin ID
if(is.null(dfOld)) {
dfOut <- dfNew # Creation of the first column
} else {
dfOut <- merge(dfOld, dfNew, by = "DatesR", all = TRUE) # Merge the new columns respecting to date column
}
return(dfOut)
}
for(id in griwrm$id) {
url <-
file.path(
"https://stratus.irstea.fr/d/0b18e688851a45478f7a/files/?p=/climaware_hydro/Q_OBS_NAT",
paste0(id, "_NAT.txt&dl=1")
)
ts <- read.table(url, sep = ";", skip = 16, header = TRUE)
# Date conversion to POSIX
ts$Date <- as.POSIXct(as.character(ts$Date), format = "%Y%m%d")
# Ptot column is merged into Precip dataframe
Precip <- MergeTS(Precip, id, ts[,c("Date", "Ptot")])
# ETP column is merged into PotEvap dataframe
PotEvap <- MergeTS(PotEvap, id, ts[,c("Date", "ETP")])
# Convert Qobs from m3/s to mm/time step
ts$Qnat <- ts$Qnat * 86.4 / griwrm$area[griwrm$id == id]
# Setting data gaps to NA
ts$Qnat[ts$Qnat <= 0] <- NA
# Qnat column is merged into Qobs dataframe
Qobs <- MergeTS(Qobs, id, ts[,c("Date", "Qnat")])
}
DatesR <- Precip$DatesR
Precip$DatesR <- NULL
PotEvap$DatesR <- NULL
Qobs$DatesR <- NULL
The daily mean precipitation and potential evaporation at the scale of the intermediate sub-basins are extracted from the SAFRAN reanalysis [@vidal50yearHighresolutionAtmospheric2010].
The daily naturalised flow is provided by @hydratecActualisationBaseDonnees2011a.
These data are embedded in a the R package 'seinebasin' which is not publicly available.
```{r QNAT, warning=FALSE, message=FALSE}
library(seinebasin)
data(QNAT)
```
## Generate the GRIWRM InputsModel object
......@@ -108,17 +79,17 @@ The GRIWRM InputsModel object is a list of **airGR** InputsModel. The identifier
The **airGR** CreateInputsModel function is extended in order to handle the griwrm object which describe the basin diagram:
```{r}
InputsModel <- CreateInputsModel(griwrm, DatesR, Precip, PotEvap, Qobs)
```{r CreateInputsModel}
InputsModel <- CreateInputsModel(griwrm, DatesR, Precip, PotEvap, Qnat)
```
## Save data for next vignettes
```{r}
```{r save}
dir.create("_cache", showWarnings = FALSE)
save(griwrm, Qobs, InputsModel, file = "_cache/V01.RData")
save(griwrm, DatesR, Precip, PotEvap, Qobs, MergeTS, file = "_cache/V01b.RData")
save(griwrm, InputsModel, file = "_cache/V01.RData")
```
# References
......@@ -13,7 +13,7 @@ knitr::opts_chunk$set(echo = TRUE)
## Load libraries
```{r}
```{r library}
library(airGRiwrm)
```
......@@ -24,7 +24,7 @@ library(airGRiwrm)
Run `vignette("01_First_network", package = "airGRiwrm")` before this one in order to create the Rdata file loaded below:
```{r}
```{r load}
load("_cache/V01.RData")
```
......@@ -32,11 +32,9 @@ load("_cache/V01.RData")
Data comes from calibration of ClimAware project with naturalised flows.
```{r}
ClimAwareParams <- read.csv(
file = "https://stratus.irstea.fr/d/0b18e688851a45478f7a/files/?p=/climaware_hydro/Ident_NAT_P0P0.txt&dl=1"
)
```{r ClimAwareParams}
library(seinebasin)
data(ClimAwareParams)
ClimAwareParams
```
......@@ -45,7 +43,7 @@ Almost all sub basin has only a lag parameter. The only exception is for La Marn
This lag parameter has to be converted in a speed in m/s used in the **airGR** lag model:
```{r}
```{r ParamClimAware}
# Convert TGR routing parameter into speed
params <- merge(griwrm, ClimAwareParams, by.x = "id", by.y = "id_sgl")
......@@ -77,7 +75,7 @@ The user must at least define the following arguments:
* IndPeriod_Run: the period on which the model is run
```{r}
```{r IndPeriod_Run}
IndPeriod_Run <- seq(
which(InputsModel[[1]]$DatesR == (InputsModel[[1]]$DatesR[1] + 365*24*60*60)), # Set aside warm-up period
length(InputsModel[[1]]$DatesR) # Until the end of the time series
......@@ -86,13 +84,13 @@ IndPeriod_Run <- seq(
The warmup period could also be defined as is:
```{r}
```{r IndPeriod_WarmUp}
IndPeriod_WarmUp = seq(1,IndPeriod_Run[1]-1)
```
```{r}
```{r CreateRunOptions}
RunOptions <- CreateRunOptions(
InputsModel = InputsModel,
IndPeriod_WarmUp = IndPeriod_WarmUp,
......@@ -104,7 +102,7 @@ RunOptions <- CreateRunOptions(
## Run the SD model for the whole basin
```{r}
```{r RunModel}
OutputsModelsClimAware <- RunModel(
InputsModel,
RunOptions = RunOptions,
......@@ -114,14 +112,15 @@ OutputsModelsClimAware <- RunModel(
## Save data for next vignettes
```{r}
```{r save}
save(RunOptions, ParamClimAware, IndPeriod_Run, file = "_cache/V02.RData")
```
## Plot the result for each basin
```{r, fig.height = 5, fig.width = 8}
plot(OutputsModelsClimAware, Qobs = Qobs[IndPeriod_Run,])
```{r plot, fig.height = 5, fig.width = 8}
data(QNAT)
plot(OutputsModelsClimAware, Qobs = Qnat[IndPeriod_Run,])
```
......
......@@ -15,44 +15,41 @@ knitr::opts_chunk$set(echo = TRUE)
Run `vignette("01_First_network", package = "airGRiwrm")` and `vignette("02_First_run", package = "airGRiwrm")` before this one in order to create the Rdata files loaded below:
```{r}
```{r load}
library(airGRiwrm)
load("_cache/V01.RData")
load("_cache/V02.RData")
library(seinebasin)
data(QNAT)
```
## InputsCrit object
```{r}
```{r CreateInputsCrit}
InputsCrit <- CreateInputsCrit(
InputsModel = InputsModel,
FUN_CRIT = airGR::ErrorCrit_KGE2,
RunOptions = RunOptions, Obs = Qobs[IndPeriod_Run,]
RunOptions = RunOptions, Obs = Qnat[IndPeriod_Run,]
)
str(InputsCrit)
```
## GRiwrmCalibOptions object
```{r}
```{r CreateCalibOptions}
CalibOptions <- CreateCalibOptions(InputsModel)
str(CalibOptions)
```
## Calibration
```{r}
```{r Calibration}
OutputsCalib <- Calibration(InputsModel, RunOptions, InputsCrit, CalibOptions)
```
```{r}
save(OutputsCalib, file = "_cache/V03.RData")
```
## Run model with Michel calibration
```{r}
```{r RunModel}
ParamMichel <- sapply(griwrm$id, function(x) {OutputsCalib[[x]]$Param})
OutputsModels <- RunModel(
......@@ -64,14 +61,14 @@ OutputsModels <- RunModel(
## Save calibration data for next vignettes
```{r}
```{r save}
save(ParamMichel, file = "_cache/V03.RData")
```
## Plot the result for each basin
```{r, fig.height = 5, fig.width = 8}
plot(OutputsModels, Qobs = Qobs[IndPeriod_Run,])
```{r plot, fig.height = 5, fig.width = 8}
plot(OutputsModels, Qobs = Qnat[IndPeriod_Run,])
```
......@@ -6,27 +6,24 @@ vignette: >
%\VignetteIndexEntry{Seine_04: Running open-loop influenced flow semi-distributed model network}
%\VignetteEngine{knitr::rmarkdown}
%\VignetteEncoding{UTF-8}
bibliography: seinebasin.bib
---
```{r, include = FALSE}
```{r setup, include = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>"
)
library(DiagrammeR)
```
```{r setup}
```{r library}
library(airGRiwrm)
library(DiagrammeR)
```
## Integration of the reservoir connections into the model
### Loading naturalised data
```{r}
load("_cache/V01b.RData")
```
The goal of this vignette is to add reservoirs connections to the Seine River model previously set up. A complete description of the reservoir system of the Seine River can be found in @dehayEtudeImpactChangement2012
### Add connections to the reservoirs in the gauging station network
......@@ -34,7 +31,7 @@ Because of the **airGR** SD model structure, the topology of the network will di
For example, the physical topology of the Aube lake is represented below:
```{r echo = FALSE, fig.height = 3}
```{r mmd_aube_true, echo = FALSE, fig.height = 3}
mmd <- function(x, ...) {
# For avoiding crash of R CMD build in console mode
......@@ -57,7 +54,7 @@ style AUBE fill:#ccf
In the SD model, we do not model intermediate flows as well as the reservoir in the catchment. As a result, an equivalent topology compatible with **airGR** will be the one below:
```{r echo = FALSE}
```{r mmd_aube_model, echo = FALSE}
mmd("
graph LR
TRANN_01 -->|68km| ARCIS_24
......@@ -72,7 +69,7 @@ style AUBE_R3 fill:#fcc
Configuration on lake Seine is similar:
```{r echo = FALSE, fig.height = 3}
```{r mmd_seine_true, echo = FALSE, fig.height = 3}
mmd("
graph LR
BAR-S_06 -->|6km| SEINE_P7
......@@ -88,7 +85,7 @@ graph LR
which can be translate as:
```{r echo = FALSE}
```{r mmd_seine_model, echo = FALSE}
mmd("
graph LR
BAR-S_06 -->|79.7km| MERY-_22
......@@ -101,7 +98,7 @@ graph LR
Pannecière is an inline reservoir:
```{r echo = FALSE, fig.height = 2}
```{r mmd_pannec_true, echo = FALSE, fig.height = 2}
mmd("
graph LR
P(PANNECIERE)
......@@ -113,7 +110,7 @@ style P fill:#ccf
It can be modelled as:
```{r echo = FALSE}
```{r mmd_pannec_model, echo = FALSE}
mmd("
graph LR
CHAUM_07 --> |153km| GURGY_02
......@@ -128,7 +125,7 @@ With $Q_{PANNEC\_P} = - Q_{CHAUM\_07}$ as all the upstream flow fills the reserv
Marne lake can be mapped as:
```{r echo = FALSE}
```{r mmd_marne_true, echo = FALSE}
mmd("
graph LR
LOUVE_19 -->|0.5km| MARNE_P28
......@@ -147,7 +144,7 @@ style M fill:#ccf
```
And can be modeled as:
```{r echo = FALSE}
```{r mmd_marne_model, echo = FALSE}
mmd("
graph LR
MARNE_P23 -->|3km| STDIZ_04
......@@ -163,7 +160,7 @@ style MARNE_R25 fill:#fcc
Hence the topological connection to the reservoirs is described in the model as below:
```{r}
```{r reservoir_connections}
reservoir_connections <- read.table(
file = system.file("seine_data", "network_reservoir_connections.txt", package = "seinebasin"),
sep = ";", header = TRUE
......@@ -173,10 +170,11 @@ reservoir_connections
Reservoir connections are added to the GRiwrm object:
```{r}
```{r griwrm2}
reservoir_connections$length <- reservoir_connections$length / 1000
reservoir_connections$model <- NA
reservoir_connections$area <- NA
load("_cache/V01.RData")
griwrm2 <- rbind(griwrm, reservoir_connections[, names(griwrm)])
DiagramGRiwrm(griwrm2)
```
......@@ -186,41 +184,18 @@ DiagramGRiwrm(griwrm2)
Description of the files, the columns and the type of connection (inlet / outlet) are defined in the list below:
```{r}
```{r lCfgReservoirs}
lCfgReservoirs <- jsonlite::read_json(system.file("seine_data", "config_reservoirs.json", package = "seinebasin"))
str(lCfgReservoirs)
```
Then, we load observation data for each reservoir and each connection:
```{r}
for(reservoir_id in names(lCfgReservoirs)) {
df <- read.table(
file = file.path(
"https://stratus.irstea.fr/d/0b18e688851a45478f7a/files/?p=/Reservoir_current_rules",
paste0(lCfgReservoirs[[reservoir_id]]$file, "&dl=1")
),
sep = "\t", header = TRUE, fill = TRUE
)
df$DatesR <- as.POSIXct(df$Date, format = "%d/%m/%Y")
Qreservoir <- merge(data.frame(DatesR = DatesR), df, by = "DatesR", all.x = TRUE)
for(connect in names(lCfgReservoirs[[reservoir_id]]$connections)) {
message("Processing ", connect)
# Replace data gap by zeros (should be interpolated ?)
Qreservoir[is.na(Qreservoir[,lCfgReservoirs[[reservoir_id]]$connections[[connect]]$col]),lCfgReservoirs[[reservoir_id]]$connections[[connect]]$col] <- 0
if(lCfgReservoirs[[reservoir_id]]$connections[[connect]]$type == "in") {
# Reservoir inlet flow is a withdrawal for the catchment: this is a negative flow to route
Qreservoir[,lCfgReservoirs[[reservoir_id]]$connections[[connect]]$col] <- -Qreservoir[,lCfgReservoirs[[reservoir_id]]$connections[[connect]]$col]
}
Qobs <- cbind(
Qobs,
Qreservoir[,lCfgReservoirs[[reservoir_id]]$connections[[connect]]$col] * 86400
)
names(Qobs)[ncol(Qobs)] <- connect
}
}
```{r seinebasin}
library(seinebasin)
data(QNAT)
data(Qreservoirs)
Qnat <- cbind(Qnat, Qreservoirs)
```
## How to handle online reservoir? The Pannecière lake case.
......@@ -232,22 +207,23 @@ There are two possibilities:
If we know in advance the flow released by the reservoir, upstream flow informations is not usefull for the simulation. But reservoir management simulation would need upstream flow informations for simulating the reservoir state during simulation. The second alternative will be useful for the next phases.
```{r}
Qobs$PANNEC_P <- -Qobs$CHAUM_07 * griwrm2$area[griwrm2$id == "CHAUM_07"] * 1e3
```{r Qnat}
Qnat <- cbind(Qnat, -Qnat[,"CHAUM_07"] * griwrm2$area[griwrm2$id == "CHAUM_07"] * 1e3)
colnames(Qnat)[ncol(Qnat)] <- "PANNEC_P"
```
## Create the InputsModel object
```{r}
InputsModel2 <- CreateInputsModel(griwrm2, DatesR, Precip, PotEvap, Qobs)
```{r CreateInputsModel}
InputsModel2 <- CreateInputsModel(griwrm2, DatesR, Precip, PotEvap, Qnat)
```
## Run simulation with naturalised flow parameters
### Load calibration parameters
```{r}
```{r load}
# Load RunOptions
load("_cache/V02.RData")
# Load calibrated parameters with Michel's method
......@@ -259,14 +235,14 @@ load("_cache/V03.RData")
A lag parameter is now mandatory for these sub-basins. As no calibration is possible at that stage an arbitrary one will be used.
```{r}
```{r ParamMichel}
ParamMichel$STDIZ_04 <- c(1, ParamMichel$STDIZ_04)
```
### Run simulation
```{r}
```{r RunModel}
OutputsModels2 <- RunModel(
InputsModel2,
RunOptions = RunOptions,
......@@ -278,36 +254,12 @@ OutputsModels2 <- RunModel(
### Load observed flow
```{r, warning=FALSE, message=FALSE}
Qobs2 <- NULL
# Files of observed flows doesn't exist for some basins
NatIds <- c("CHAUM_07", "MONTE_15", "NOGEN_13", "STGER_09", "TRANN_01")
ids <- griwrm$id[!griwrm$id %in% NatIds]
for(id in ids) {
url <-
file.path(
"https://stratus.irstea.fr/d/0b18e688851a45478f7a/files/?p=/climaware_hydro/Q_OBS_NAT",
paste0(id, "_BV.txt&dl=1")
)
ts <- read.table(file = url,
sep = ";", skip = 24, header = TRUE)
# Date conversion to POSIX
ts$Date <- as.POSIXct(as.character(ts$Date), format = "%Y%m%d")
# Convert Qobs from l/s to mm/time step
ts$Q <- ts$Q * 86.4 / griwrm$area[griwrm$id == id] / 1000
# Setting data gaps to NA
ts$Q[ts$Q <= 0] <- NA
# Column is merged into Qobs dataframe
Qobs2 <- MergeTS(Qobs2, id, ts[,c("Date", "Q")])
}
```{r QOBS, warning=FALSE, message=FALSE}
data(QOBS)
```
## Save data for next vignettes
```{r}
```{r save}
ReduceOutputsModel <- function(OutputsModels, IndPeriod) {
items <- names(OutputsModels)
OutputsModelsOut <- sapply(items, function(x) {OutputsModels[[x]] <- OutputsModels[[x]][IndPeriod]})
......@@ -316,17 +268,18 @@ ReduceOutputsModel <- function(OutputsModels, IndPeriod) {
return(OutputsModelsOut)
}
QobsReservoirs <- Qobs[DatesR < "2008-08-01", reservoir_connections$id]
save(QobsReservoirs, Qobs2, griwrm2, ReduceOutputsModel, file = "_cache/V04.RData")
save(griwrm2, ReduceOutputsModel, file = "_cache/V04.RData")
```
### Comparison with simulated flows
```{r, fig.height = 5, fig.width = 8}
```{r plot, fig.height = 5, fig.width = 8}
IndPeriod <- RunOptions[[1]]$IndPeriod_Run
IndPeriod <- IndPeriod[IndPeriod <= nrow(Qobs)]
htmltools::tagList(lapply(
ids,
colnames(Qobs),
function(x) {
Q2 <- Qobs2[RunOptions[[1]]$IndPeriod_Run,x]
Q2 <- Qobs[IndPeriod,x]
IndPeriod_Obs <- which(!is.na(Q2))
OutputsModels <- ReduceOutputsModel(OutputsModels2[[x]], IndPeriod_Obs)
plot(OutputsModels, Qobs = Q2[IndPeriod_Obs], main = x, which = c("Regime"))
......@@ -334,5 +287,4 @@ htmltools::tagList(lapply(
))
```
# References
......@@ -8,31 +8,32 @@ vignette: >
%\VignetteEncoding{UTF-8}
---
```{r, include = FALSE}
```{r setup, include = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>"
)
```
```{r setup}
```{r library}
library(airGRiwrm)
```
This vignette proposes an example of calibration of influenced flow with the Marne reservoir. It will used influenced observation flows directly measured at gauging stations and flows recorded at reservoir inlets and outlet.
This vignette proposes an example of calibration of influenced flow with the Marne reservoir.
It will used influenced observation flows directly measured at gauging stations and flows recorded at reservoir inlets and outlet.
## Set the data
Loading naturalised data and influenced flow configuration:
```{r}
load("_cache/V01b.RData")
```{r load}
#load("_cache/V01.RData")
load("_cache/V04.RData")
```
Remove e