xcmsSet - old version (rev: 15646e937936) - conda version resolving never finishes

[Lain-inrae]

Hello,
I'm trying to re-deploy an old version of xcmsSet (from 2017 - 15646e937936) in a new galaxy (release_21.09):

{
  "tool_panel_section_id": "",
  "tool_shed_url": "https://toolshed.g2.bx.psu.edu/",
  "name": "xcms_xcmsset",
  "owner": "lecorguille",
  "changeset_revision": "15646e937936",
  "new_tool_panel_section_label": "",
  "shed_tool_conf": "/galaxy/config/toolshed_metabolomics_tool_conf.xml",
  "install_resolver_dependencies": true,
  "install_tool_dependencies": true,
  "install_repository_dependencies": true
}

When galaxy tries to install this tool at this specific version, it runs the following:

/galaxy/database/dependencies/_conda/bin/conda create -y --quiet --override-channels --channel conda-forge --channel bioconda --channel defaults --name mulled-v1-muled-number r-snow=0.4_1 bioconductor-xcms=1.46.0 r-batch=1.1_4

This command is stuck at the SAT stage of conda (versions conflicts resolving, see this SO post), even with a fresh new install (nothing's installed yet).
FYI, r-snow in only available at version 0.4_4

When we don't fix neither batch nor snow versions, I get:

/galaxy/database/dependencies/_conda/bin/conda create -y --quiet --override-channels --channel conda-forge --channel bioconda --channel defaults -v -v --name mulled-test-no-any-version r-snow bioconductor-xcms=1.46.0 r-batch

TL;DR:
r-snow conda-forge/noarch::r-snow-0.4_4-r41hc72bb7e_0
r-batch conda-forge/noarch::r-batch-1.1_5-r41hc72bb7e_2

The following NEW packages will be INSTALLED:

  _libgcc_mutex      conda-forge/linux-64::_libgcc_mutex-0.1-conda_forge
  _openmp_mutex      conda-forge/linux-64::_openmp_mutex-4.5-1_gnu
  _r-mutex           conda-forge/noarch::_r-mutex-1.0.1-anacondar_1
  binutils_impl_lin~ conda-forge/linux-64::binutils_impl_linux-64-2.36.1-h193b22a_2
  binutils_linux-64  conda-forge/linux-64::binutils_linux-64-2.36-hf3e587d_6
  bioconductor-biob~ bioconda/linux-64::bioconductor-biobase-2.54.0-r41h5c21468_1
  bioconductor-bioc~ bioconda/noarch::bioconductor-biocgenerics-0.40.0-r41hdfd78af_0
  bioconductor-mzr   bioconda/linux-64::bioconductor-mzr-2.28.0-r41h619a076_1
  bioconductor-prot~ bioconda/noarch::bioconductor-protgenerics-1.26.0-r41hdfd78af_0
  bioconductor-rhdf~ bioconda/linux-64::bioconductor-rhdf5lib-1.16.0-r41h5c21468_1
  bioconductor-xcms  bioconda/linux-64::bioconductor-xcms-1.46.0-0
  bioconductor-zlib~ bioconda/linux-64::bioconductor-zlibbioc-1.40.0-r41h5c21468_1
  bwidget            conda-forge/linux-64::bwidget-1.9.14-ha770c72_1
  bzip2              conda-forge/linux-64::bzip2-1.0.8-h7f98852_4
  c-ares             conda-forge/linux-64::c-ares-1.18.1-h7f98852_0
  ca-certificates    conda-forge/linux-64::ca-certificates-2021.10.8-ha878542_0
  cairo              conda-forge/linux-64::cairo-1.16.0-ha00ac49_1009
  curl               conda-forge/linux-64::curl-7.82.0-h2283fc2_0
  expat              conda-forge/linux-64::expat-2.4.7-h27087fc_0
  font-ttf-dejavu-s~ conda-forge/noarch::font-ttf-dejavu-sans-mono-2.37-hab24e00_0
  font-ttf-inconsol~ conda-forge/noarch::font-ttf-inconsolata-3.000-h77eed37_0
  font-ttf-source-c~ conda-forge/noarch::font-ttf-source-code-pro-2.038-h77eed37_0
  font-ttf-ubuntu    conda-forge/noarch::font-ttf-ubuntu-0.83-hab24e00_0
  fontconfig         conda-forge/linux-64::fontconfig-2.13.96-h8e229c2_1
  fonts-conda-ecosy~ conda-forge/noarch::fonts-conda-ecosystem-1-0
  fonts-conda-forge  conda-forge/noarch::fonts-conda-forge-1-0
  freetype           conda-forge/linux-64::freetype-2.10.4-h0708190_1
  fribidi            conda-forge/linux-64::fribidi-1.0.10-h36c2ea0_0
  gcc_impl_linux-64  conda-forge/linux-64::gcc_impl_linux-64-9.4.0-h03d3576_13
  gcc_linux-64       conda-forge/linux-64::gcc_linux-64-9.4.0-h391b98a_6
  gettext            conda-forge/linux-64::gettext-0.19.8.1-h73d1719_1008
  gfortran_impl_lin~ conda-forge/linux-64::gfortran_impl_linux-64-9.4.0-h0003116_13
  gfortran_linux-64  conda-forge/linux-64::gfortran_linux-64-9.4.0-hf0ab688_6
  graphite2          conda-forge/linux-64::graphite2-1.3.13-h58526e2_1001
  gsl                conda-forge/linux-64::gsl-2.7-he838d99_0
  gxx_impl_linux-64  conda-forge/linux-64::gxx_impl_linux-64-9.4.0-h03d3576_13
  gxx_linux-64       conda-forge/linux-64::gxx_linux-64-9.4.0-h0316aca_6
  harfbuzz           conda-forge/linux-64::harfbuzz-3.4.0-hb4a5f5f_0
  hdf4               conda-forge/linux-64::hdf4-4.2.15-h10796ff_3
  hdf5               conda-forge/linux-64::hdf5-1.12.1-nompi_h4df4325_104
  icu                conda-forge/linux-64::icu-69.1-h9c3ff4c_0
  jbig               conda-forge/linux-64::jbig-2.1-h7f98852_2003
  jpeg               conda-forge/linux-64::jpeg-9e-h7f98852_0
  kernel-headers_li~ conda-forge/noarch::kernel-headers_linux-64-2.6.32-he073ed8_15
  keyutils           conda-forge/linux-64::keyutils-1.6.1-h166bdaf_0
  krb5               conda-forge/linux-64::krb5-1.19.2-h08a2579_4
  ld_impl_linux-64   conda-forge/linux-64::ld_impl_linux-64-2.36.1-hea4e1c9_2
  lerc               conda-forge/linux-64::lerc-3.0-h9c3ff4c_0
  libblas            conda-forge/linux-64::libblas-3.9.0-13_linux64_openblas
  libcblas           conda-forge/linux-64::libcblas-3.9.0-13_linux64_openblas
  libcurl            conda-forge/linux-64::libcurl-7.82.0-h2283fc2_0
  libdeflate         conda-forge/linux-64::libdeflate-1.10-h7f98852_0
  libedit            conda-forge/linux-64::libedit-3.1.20191231-he28a2e2_2
  libev              conda-forge/linux-64::libev-4.33-h516909a_1
  libffi             conda-forge/linux-64::libffi-3.4.2-h7f98852_5
  libgcc-devel_linu~ conda-forge/linux-64::libgcc-devel_linux-64-9.4.0-hd854feb_13
  libgcc-ng          conda-forge/linux-64::libgcc-ng-11.2.0-h1d223b6_13
  libgfortran-ng     conda-forge/linux-64::libgfortran-ng-11.2.0-h69a702a_13
  libgfortran5       conda-forge/linux-64::libgfortran5-11.2.0-h5c6108e_13
  libglib            conda-forge/linux-64::libglib-2.70.2-h174f98d_4
  libgomp            conda-forge/linux-64::libgomp-11.2.0-h1d223b6_13
  libiconv           conda-forge/linux-64::libiconv-1.16-h516909a_0
  liblapack          conda-forge/linux-64::liblapack-3.9.0-13_linux64_openblas
  libnetcdf          conda-forge/linux-64::libnetcdf-4.8.1-nompi_hb3fd0d9_101
  libnghttp2         conda-forge/linux-64::libnghttp2-1.47.0-he49606f_0
  libopenblas        conda-forge/linux-64::libopenblas-0.3.18-pthreads_h8fe5266_0
  libpng             conda-forge/linux-64::libpng-1.6.37-h21135ba_2
  libsanitizer       conda-forge/linux-64::libsanitizer-9.4.0-h79bfe98_13
  libssh2            conda-forge/linux-64::libssh2-1.10.0-ha35d2d1_2
  libstdcxx-devel_l~ conda-forge/linux-64::libstdcxx-devel_linux-64-9.4.0-hd854feb_13
  libstdcxx-ng       conda-forge/linux-64::libstdcxx-ng-11.2.0-he4da1e4_13
  libtiff            conda-forge/linux-64::libtiff-4.3.0-h542a066_3
  libuuid            conda-forge/linux-64::libuuid-2.32.1-h7f98852_1000
  libwebp-base       conda-forge/linux-64::libwebp-base-1.2.2-h7f98852_1
  libxcb             conda-forge/linux-64::libxcb-1.13-h7f98852_1004
  libxml2            conda-forge/linux-64::libxml2-2.9.12-h885dcf4_1
  libzip             conda-forge/linux-64::libzip-1.8.0-h1c5bbd1_1
  libzlib            conda-forge/linux-64::libzlib-1.2.11-h36c2ea0_1013
  lz4-c              conda-forge/linux-64::lz4-c-1.9.3-h9c3ff4c_1
  make               conda-forge/linux-64::make-4.3-hd18ef5c_1
  ncurses            conda-forge/linux-64::ncurses-6.2-h58526e2_4
  openssl            conda-forge/linux-64::openssl-3.0.0-h7f98852_2
  pango              conda-forge/linux-64::pango-1.50.5-h4dcc4a0_0
  pcre               conda-forge/linux-64::pcre-8.45-h9c3ff4c_0
  pcre2              conda-forge/linux-64::pcre2-10.37-h032f7d1_0
  pixman             conda-forge/linux-64::pixman-0.40.0-h36c2ea0_0
  pthread-stubs      conda-forge/linux-64::pthread-stubs-0.4-h36c2ea0_1001
  r                  conda-forge/noarch::r-4.1-r41hd8ed1ab_1004
  r-base             conda-forge/linux-64::r-base-4.1.2-h2553ce4_1
  r-batch            conda-forge/noarch::r-batch-1.1_5-r41hc72bb7e_2
  r-boot             conda-forge/noarch::r-boot-1.3_28-r41hc72bb7e_0
  r-class            conda-forge/linux-64::r-class-7.3_20-r41hcfec24a_0
  r-cluster          conda-forge/linux-64::r-cluster-2.1.2-r41h859d828_0
  r-codetools        conda-forge/noarch::r-codetools-0.2_18-r41hc72bb7e_0
  r-foreign          conda-forge/linux-64::r-foreign-0.8_82-r41hcfec24a_0
  r-kernsmooth       conda-forge/linux-64::r-kernsmooth-2.23_20-r41h742201e_0
  r-lattice          conda-forge/linux-64::r-lattice-0.20_45-r41hcfec24a_0
  r-mass             conda-forge/linux-64::r-mass-7.3_55-r41hcfec24a_0
  r-matrix           conda-forge/linux-64::r-matrix-1.4_0-r41he454529_0
  r-mgcv             conda-forge/linux-64::r-mgcv-1.8_39-r41h0154571_0
  r-ncdf4            conda-forge/linux-64::r-ncdf4-1.19-r41h186726c_0
  r-nlme             conda-forge/linux-64::r-nlme-3.1_155-r41h859d828_0
  r-nnet             conda-forge/linux-64::r-nnet-7.3_17-r41hcfec24a_0
  r-rcolorbrewer     conda-forge/noarch::r-rcolorbrewer-1.1_2-r41h785f33e_1003
  r-rcpp             conda-forge/linux-64::r-rcpp-1.0.8-r41h03ef668_0
  r-recommended      conda-forge/noarch::r-recommended-4.1-r41hd8ed1ab_1004
  r-rpart            conda-forge/linux-64::r-rpart-4.1.16-r41hcfec24a_0
  r-snow             conda-forge/noarch::r-snow-0.4_4-r41hc72bb7e_0
  r-spatial          conda-forge/linux-64::r-spatial-7.3_15-r41hcfec24a_0
  r-survival         conda-forge/linux-64::r-survival-3.3_1-r41h06615bd_0
  readline           conda-forge/linux-64::readline-8.1-h46c0cb4_0
  sed                conda-forge/linux-64::sed-4.8-he412f7d_0
  sysroot_linux-64   conda-forge/noarch::sysroot_linux-64-2.12-he073ed8_15
  tk                 conda-forge/linux-64::tk-8.6.12-h27826a3_0
  tktable            conda-forge/linux-64::tktable-2.10-hb7b940f_3
  xorg-kbproto       conda-forge/linux-64::xorg-kbproto-1.0.7-h7f98852_1002
  xorg-libice        conda-forge/linux-64::xorg-libice-1.0.10-h7f98852_0
  xorg-libsm         conda-forge/linux-64::xorg-libsm-1.2.3-hd9c2040_1000
  xorg-libx11        conda-forge/linux-64::xorg-libx11-1.7.2-h7f98852_0
  xorg-libxau        conda-forge/linux-64::xorg-libxau-1.0.9-h7f98852_0
  xorg-libxdmcp      conda-forge/linux-64::xorg-libxdmcp-1.1.3-h7f98852_0
  xorg-libxext       conda-forge/linux-64::xorg-libxext-1.3.4-h7f98852_1
  xorg-libxrender    conda-forge/linux-64::xorg-libxrender-0.9.10-h7f98852_1003
  xorg-libxt         conda-forge/linux-64::xorg-libxt-1.2.1-h7f98852_2
  xorg-renderproto   conda-forge/linux-64::xorg-renderproto-0.11.1-h7f98852_1002
  xorg-xextproto     conda-forge/linux-64::xorg-xextproto-7.3.0-h7f98852_1002
  xorg-xproto        conda-forge/linux-64::xorg-xproto-7.0.31-h7f98852_1007
  xz                 conda-forge/linux-64::xz-5.2.5-h516909a_1
  zlib               conda-forge/linux-64::zlib-1.2.11-h36c2ea0_1013
  zstd               conda-forge/linux-64::zstd-1.5.2-ha95c52a_0


Preparing transaction: ...working... DEBUG conda.core.link:_get_python_version(724): no python version found in prefix
done
Verifying transaction: ...working... done

...

done

When we fix either snow of batch version, conda never finishes (I let them run 30 minutes before killing them - stuck at SAT):

/galaxy/database/dependencies/_conda/bin/conda create -y --quiet --override-channels --channel conda-forge --channel bioconda --channel defaults -v -v --name mulled-test-no-snow-version r-snow bioconductor-xcms=1.46.0 r-batch=1.1_4

/galaxy/database/dependencies/_conda/bin/conda create -y --quiet --override-channels --channel conda-forge --channel bioconda --channel defaults -v -v --name mulled-test-no-batch-version r-snow=0.4_1 bioconductor-xcms=1.46.0 r-batch

Am I doing something wrong..?
Thanks!

[lecorguille]

What is the issue with the solution without the version of both snow and batch?

[Lain-inrae]

There are no problem with this solution (I think, I have not tested the tool).
The problem is: when galaxy tries to install the tool, we don't have the choice to use other versions than the ones described in the abims_xcms_xcmsSet.xml

[Lain-inrae]

If I don't have the choice, I will install it myself - I will bypass galaxy's installation process.
But that makes the deployment of our galaxy (and it tools) non reproducible (unless I write a script for those tools).

[bgruening]

Those are very old conda packages, where conda still was not pinning R correctly. We either need to update those old R packages, upgrade the Galaxy tool to use more recent packages or you can use the BioContainer that hopefully exists for this tool.

[Lain-inrae]

Hello,
I come back after some investigation about r-batch.
This package has not not been maintained since 2018. It is - in a majority of cases - used for command line parsing.
The most simple usage of this package R -q -e "batch::parseCommandArgs()" is very dangerous as it is prone to RCE (I managed to deploy a simple backdoor in python).
There are three packages that may replace it for the command line parsing. These packages are well maintained and I suppose they are well tested against command injections:
https://stackoverflow.com/questions/3433603/parsing-command-line-arguments-in-r-scripts

There are three packages on CRAN:

getopt: C-like getopt behavior
optparse: a command line parser inspired by Python's optparse library
argparse: a command line optional and positional argument parser (inspired by Python's argparse library). This package requires that a Python interpreter be installed with the argparse and json (or simplejson)