Skip to content
GitLab
Projects Groups Topics Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Register
  • Sign in
  • evalhyd-r evalhyd-r
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributor statistics
    • Graph
    • Compare revisions
  • Issues 3
    • Issues 3
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 0
    • Merge requests 0
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Terraform modules
  • Monitor
    • Monitor
    • Metrics
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar

La forge institutionnelle d'INRAE étant en production depuis le 10 juin 2025, nous vous invitons à y créer vos nouveaux projets.

  • HYCAR-HydroHYCAR-Hydro
  • evalhydevalhyd
  • evalhyd-revalhyd-r
  • Issues
  • #4
Closed
Open
Issue created May 15, 2023 by Thibault Hallouin@thibault.hallouinOwner

memory leak with metrics relying on `xt::sort` or `xt::quantile`

When using the xtensor sorting functions (xt::sort and xt::quantile) on rtensor, there is a memory leak, as some of the allocated memory seems not to be released between calls.

For example, if one uses any of these two functions repeatedly:

// [[Rcpp::export("sort_cpp")]]
xt::rarray<double> sort_cpp(
    xt::rtensor<double, 4> q
)
{
  xt::rarray<double> arr = xt::sort(q, 3);
  return arr;
}

// [[Rcpp::export("quantile_cpp")]]
xt::rarray<double> quantile_cpp(
    xt::rtensor<double, 4> q
)
{
  xt::rarray<double> arr = xt::quantile(q, {0.25}, 2);
  return arr;
}

Then, the memory used by R will gradually increase (even with an explicit garbage collection):

q = array(
  data=rep(c(0.6), 10 * 2 * 11 * 2080),
  dim=c(10, 2, 11, 2080)
)

for (i in 1:1000) 
{
  myTestLib::sort_cpp(q)
  gc()
}

for (i in 1:1000) 
{
  myTestLib::quantile_cpp(q)
  gc()
}

All evalhyd metrics relying any of these sorting functions are impacted by the memory leak, i.e. QS, CRPS_FROM_QS, CRPS_FROM_ECDF, CRPS_FROM_BS, AW, AWI, AWN, WS, WSS, RANK_HIST, DS, AS. Depending on the size of the array it is working on, the impact may be more critical for some metrics in particular.

And, the memory seems to only be released when the R session is shut down.

Edited May 16, 2023 by Thibault Hallouin
Assignee
Assign to
Time tracking