Commit 1cabf829 authored by Dorchies David's avatar Dorchies David
Browse files

tests: try to solve false negative on execution time benchmark

- I increase the margin of acceptable performance degradation between CRAN version and tested versions of RunModel

Refs #136
parent 104d22ba
Pipeline #28613 failed with stages
in 14 minutes and 37 seconds
......@@ -3,7 +3,10 @@ df <- read.table("../tmp/mean_execution_time.tsv", sep = "\t", header = T)
lapply(df$model, function(model) {
test_that(paste(model, ": RunModel should be as fast as CRAN version"), {
sel <- df$model == model
threshold <- max(0.2, -0.15 * df[sel, 2] + 1) # decrease from 1 at 0.1ms to 0.2 at 5.5ms
# Limit threshold for evolution of execution time (in %) between the 2 versions
# Negative values of evolution are expected but we apply an error margin depending on execution time
# decrease from 1.5 at 0.0ms to 0.5 at 10ms with a minimum at 0.5
threshold <- max(0.5, -0.1 * df[sel, 2] + 1.5)
expect_lt(df$evolution[sel], threshold)
})
})
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment