memory leak with metrics relying on `xt::sort` or `xt::quantile`
When using the xtensor
sorting functions (xt::sort
and xt::quantile
) on rtensor
, there is a memory leak, as some of the allocated memory seems not to be released between calls.
For example, if one uses any of these two functions repeatedly:
// [[Rcpp::export("sort_cpp")]]
xt::rarray<double> sort_cpp(
xt::rtensor<double, 4> q
)
{
xt::rarray<double> arr = xt::sort(q, 3);
return arr;
}
// [[Rcpp::export("quantile_cpp")]]
xt::rarray<double> quantile_cpp(
xt::rtensor<double, 4> q
)
{
xt::rarray<double> arr = xt::quantile(q, {0.25}, 2);
return arr;
}
Then, the memory used by R will gradually increase (even with an explicit garbage collection):
q = array(
data=rep(c(0.6), 10 * 2 * 11 * 2080),
dim=c(10, 2, 11, 2080)
)
for (i in 1:1000)
{
myTestLib::sort_cpp(q)
gc()
}
for (i in 1:1000)
{
myTestLib::quantile_cpp(q)
gc()
}
All evalhyd
metrics relying any of these sorting functions are impacted by the memory leak, i.e. QS, CRPS_FROM_QS, CRPS_FROM_ECDF, CRPS_FROM_BS, AW, AWI, AWN, WS, WSS, RANK_HIST, DS, AS. Depending on the size of the array it is working on, the impact may be more critical for some metrics in particular.
And, the memory seems to only be released when the R session is shut down.