Describe the issue or bug

Attempting to import the r package archive using rpy2-v3.5.16 raises an rpy2.rinterface_lib.embedded.RRuntimeError exception.

from rpy2.robjects.packages import importr
archive = importr('archive')

Note that:

• updating the LD_LIBRARY_PATH, as described in the README.md does not resolve the issue.
• Performing an LD_PRELOAD trick does resolve the issue (i.e.: running LD_PRELOAD="$R_HOME/lib/libR.so" python3 -c 'from rpy2.robjects.packages import importr; archive = importr("archive")' does not raise any exception.

To Reproduce

1. Install R-package 'archive'

   R --slave -e 'install.packages("archive"); library(archive); packageVersion("archive")'

   Standard output

   Installing package into ‘/home/mlefeuvre/R/x86_64-pc-linux-gnu-library/4.1’
   (as ‘lib’ is unspecified)
   trying URL 'https://cloud.r-project.org/src/contrib/archive_1.1.9.tar.gz'
   Content type 'application/x-gzip' length 142274 bytes (138 KB)
   ==================================================
   downloaded 138 KB
    
   * installing *source* package ‘archive’ ...
   ** package ‘archive’ successfully unpacked and MD5 sums checked
   ** using staged installation
   ** Running ./configure
   Found pkg-config cflags and libs!
   PKG_CONFIG_PATH=
   PKG_CFLAGS=
   PKG_LIBS=-larchive
   ** libs
   g++ -std=gnu++14 -I"/usr/share/R/include" -DNDEBUG -Icpp11/include -I'/home/mlefeuvre/R/x86_64-pc-linux-gnu-library/4.1/cli/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-4A2Reg/r-base-4.1.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c archive.cpp -o archive.o
   g++ -std=gnu++14 -I"/usr/share/R/include" -DNDEBUG -Icpp11/include -I'/home/mlefeuvre/R/x86_64-pc-linux-gnu-library/4.1/cli/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-4A2Reg/r-base-4.1.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c archive_extract.cpp -o archive_extract.o
   g++ -std=gnu++14 -I"/usr/share/R/include" -DNDEBUG -Icpp11/include -I'/home/mlefeuvre/R/x86_64-pc-linux-gnu-library/4.1/cli/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-4A2Reg/r-base-4.1.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c archive_read.cpp -o archive_read.o
   g++ -std=gnu++14 -I"/usr/share/R/include" -DNDEBUG -Icpp11/include -I'/home/mlefeuvre/R/x86_64-pc-linux-gnu-library/4.1/cli/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-4A2Reg/r-base-4.1.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c archive_write.cpp -o archive_write.o
   g++ -std=gnu++14 -I"/usr/share/R/include" -DNDEBUG -Icpp11/include -I'/home/mlefeuvre/R/x86_64-pc-linux-gnu-library/4.1/cli/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-4A2Reg/r-base-4.1.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c archive_write_direct.cpp -o archive_write_direct.o
   g++ -std=gnu++14 -I"/usr/share/R/include" -DNDEBUG -Icpp11/include -I'/home/mlefeuvre/R/x86_64-pc-linux-gnu-library/4.1/cli/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-4A2Reg/r-base-4.1.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c archive_write_files.cpp -o archive_write_files.o
   g++ -std=gnu++14 -I"/usr/share/R/include" -DNDEBUG -Icpp11/include -I'/home/mlefeuvre/R/x86_64-pc-linux-gnu-library/4.1/cli/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-4A2Reg/r-base-4.1.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c cpp11.cpp -o cpp11.o
   g++ -std=gnu++14 -I"/usr/share/R/include" -DNDEBUG -Icpp11/include -I'/home/mlefeuvre/R/x86_64-pc-linux-gnu-library/4.1/cli/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-4A2Reg/r-base-4.1.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c r_archive.cpp -o r_archive.o
   gcc -I"/usr/share/R/include" -DNDEBUG -Icpp11/include -I'/home/mlefeuvre/R/x86_64-pc-linux-gnu-library/4.1/cli/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-4A2Reg/r-base-4.1.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c connection/connection.c -o connection/connection.o
   g++ -std=gnu++14 -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -flto=auto -Wl,-z,relro -o archive.so archive.o archive_extract.o archive_read.o archive_write.o archive_write_direct.o archive_write_files.o cpp11.o r_archive.o -larchive -L/usr/lib/R/lib -lR
   installing to /home/mlefeuvre/R/x86_64-pc-linux-gnu-library/4.1/00LOCK-archive/00new/archive/libs
   ** R
   ** inst
   ** byte-compile and prepare package for lazy loading
   ** help
   *** installing help indices
   ** building package indices
   ** testing if installed package can be loaded from temporary location
   ** checking absolute paths in shared objects and dynamic libraries
   ** testing if installed package can be loaded from final location
   ** testing if installed package keeps a record of temporary installation path
   * DONE (archive)
    
   The downloaded source packages are in
           ‘/tmp/RtmpwdLnlu/downloaded_packages’
   [1] ‘1.1.9’

2. Install rpy-v3.5.16

   pip3 install rpy2==3.5.16

   Standard output

   Defaulting to user installation because normal site-packages is not writeable
   Collecting rpy2==3.5.16
     Downloading rpy2-3.5.16.tar.gz (220 kB)
     Installing build dependencies ... done
     Getting requirements to build wheel ... done
     Preparing metadata (pyproject.toml) ... done
   Collecting cffi>=1.15.1 (from rpy2==3.5.16)
     Using cached cffi-1.17.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.5 kB)
   Collecting jinja2 (from rpy2==3.5.16)
     Downloading jinja2-3.1.4-py3-none-any.whl.metadata (2.6 kB)
   Collecting tzlocal (from rpy2==3.5.16)
     Downloading tzlocal-5.2-py3-none-any.whl.metadata (7.8 kB)
   Collecting pycparser (from cffi>=1.15.1->rpy2==3.5.16)
     Using cached pycparser-2.22-py3-none-any.whl.metadata (943 bytes)
   Collecting MarkupSafe>=2.0 (from jinja2->rpy2==3.5.16)
     Downloading MarkupSafe-3.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.0 kB)
   Using cached cffi-1.17.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (446 kB)
   Downloading jinja2-3.1.4-py3-none-any.whl (133 kB)
   Downloading tzlocal-5.2-py3-none-any.whl (17 kB)
   Downloading MarkupSafe-3.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (20 kB)
   Using cached pycparser-2.22-py3-none-any.whl (117 kB)
   Building wheels for collected packages: rpy2
     Building wheel for rpy2 (pyproject.toml) ... done
     Created wheel for rpy2: filename=rpy2-3.5.16-cp310-cp310-linux_x86_64.whl size=329859 sha256=267be94a97cb24d3e4762891285255fd4a0e4a02ba04abd75db12be5fc0c49e5
     Stored in directory: /home/mlefeuvre/.cache/pip/wheels/86/9b/1a/c09b2935ab01605117b270e2abb744a10f5b213c6f9b7213ce
   Successfully built rpy2
   Installing collected packages: tzlocal, pycparser, MarkupSafe, jinja2, cffi, rpy2
   Successfully installed MarkupSafe-3.0.2 cffi-1.17.1 jinja2-3.1.4 pycparser-2.22 rpy2-3.5.16 tzlocal-5.2    

3. Attempt to bind archive from python, using rpy

   python3 -c 'from rpy2.robjects.packages import importr; archive = importr("archive")'

   Standard output

   R[write to console]: Error: package or namespace load failed for ‘archive’:
    .onLoad failed in loadNamespace() for 'archive', details:
     call: dyn.load(lib_path)
     error: unable to load shared object '/home/mlefeuvre/R/x86_64-pc-linux-gnu-library/4.1/archive/lib//libconnection.so':
     /home/mlefeuvre/R/x86_64-pc-linux-gnu-library/4.1/archive/lib//libconnection.so: undefined symbol: R_new_custom_connection
    
   R[write to console]: Error: .onLoad failed in loadNamespace() for 'archive', details:
     call: dyn.load(lib_path)
     error: unable to load shared object '/home/mlefeuvre/R/x86_64-pc-linux-gnu-library/4.1/archive/lib//libconnection.so':
     /home/mlefeuvre/R/x86_64-pc-linux-gnu-library/4.1/archive/lib//libconnection.so: undefined symbol: R_new_custom_connection
    
   Traceback (most recent call last):
     File "<string>", line 1, in <module>
     File "/home/mlefeuvre/.local/lib/python3.10/site-packages/rpy2/robjects/packages.py", line 486, in importr
       env = _get_namespace(name)
     File "/home/mlefeuvre/.local/lib/python3.10/site-packages/rpy2/rinterface_lib/conversion.py", line 45, in _
       cdata = function(*args, **kwargs)
     File "/home/mlefeuvre/.local/lib/python3.10/site-packages/rpy2/rinterface.py", line 869, in __call__
       raise embedded.RRuntimeError(_rinterface._geterrmessage())
   rpy2.rinterface_lib.embedded.RRuntimeError: Error: .onLoad failed in loadNamespace() for 'archive', details:
     call: dyn.load(lib_path)
     error: unable to load shared object '/home/mlefeuvre/R/x86_64-pc-linux-gnu-library/4.1/archive/lib//libconnection.so':
     /home/mlefeuvre/R/x86_64-pc-linux-gnu-library/4.1/archive/lib//libconnection.so: undefined symbol: R_new_custom_connection

Expected behavior
Running importr("archive") should return a valid packages.Package object, without raising any error or warning.

Error

R[write to console]: Error: package or namespace load failed for ‘archive’:
 .onLoad failed in loadNamespace() for 'archive', details:
  call: dyn.load(lib_path)
  error: unable to load shared object '/home/mlefeuvre/R/x86_64-pc-linux-gnu-library/4.1/archive/lib//libconnection.so':
  /home/mlefeuvre/R/x86_64-pc-linux-gnu-library/4.1/archive/lib//libconnection.so: undefined symbol: R_new_custom_connection

R[write to console]: Error: .onLoad failed in loadNamespace() for 'archive', details:
  call: dyn.load(lib_path)
  error: unable to load shared object '/home/mlefeuvre/R/x86_64-pc-linux-gnu-library/4.1/archive/lib//libconnection.so':
  /home/mlefeuvre/R/x86_64-pc-linux-gnu-library/4.1/archive/lib//libconnection.so: undefined symbol: R_new_custom_connection

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/mlefeuvre/.local/lib/python3.10/site-packages/rpy2/robjects/packages.py", line 486, in importr
    env = _get_namespace(name)
  File "/home/mlefeuvre/.local/lib/python3.10/site-packages/rpy2/rinterface_lib/conversion.py", line 45, in _
    cdata = function(*args, **kwargs)
  File "/home/mlefeuvre/.local/lib/python3.10/site-packages/rpy2/rinterface.py", line 869, in __call__
    raise embedded.RRuntimeError(_rinterface._geterrmessage())
rpy2.rinterface_lib.embedded.RRuntimeError: Error: .onLoad failed in loadNamespace() for 'archive', details:
  call: dyn.load(lib_path)
  error: unable to load shared object '/home/mlefeuvre/R/x86_64-pc-linux-gnu-library/4.1/archive/lib//libconnection.so':
  /home/mlefeuvre/R/x86_64-pc-linux-gnu-library/4.1/archive/lib//libconnection.so: undefined symbol: R_new_custom_connection

Additional context

rpy2 version:
3.5.16
Python version:
3.10.12 (main, Sep 11 2024, 15:47:36) [GCC 11.4.0]
Looking for R's HOME:
    Environment variable R_HOME: /usr/lib/R
    Calling `R RHOME`: /usr/lib/R
    Environment variable R_LIBS_USER: None
R's value for LD_LIBRARY_PATH:
/usr/lib/R/lib:/usr/lib/x86_64-linux-gnu:/usr/lib/jvm/default-java/lib/server
R version:
    In the PATH: R version 4.1.2 (2021-11-01) -- "Bird Hippie"
    Loading R library from rpy2: OK
Additional directories to load R packages from:
None
C extension compilation:
  include:
  ['/usr/share/R/include']
  libraries:
  ['R', 'pcre2-8', 'lzma', 'bz2', 'z', 'tirpc', 'rt', 'dl', 'm', 'icuuc', 'icui18n']
  library_dirs:
  ['/usr/lib/R/lib']
  extra_compile_args:
  ['-std=c99']
  extra_link_args:
  ['-Wl,--export-dynamic', '-fopenmp', '-Wl,-Bsymbolic-functions', '-flto=auto', '-ffat-lto-objects', '-flto=auto', '-Wl,-z,relro']
Directory for the R shared library:
lib
CFFI extension type
  Environment variable: RPY2_CFFI_MODE
  Value: CFFI_MODE.ANY
  ABI: PRESENT
  API: PRESENT

Some additional remarks:

• May be related to Unable to set Python interface for R code #913 (?)
• Error state is specifically encountered here : r-lib/archive/R/zzz.R
• Issue does not seem related to any conda "shenanigans", although it was first encountered through it...
• Note that the steps required to reproduce this bug were also tested on a VirtualBox VM, using a blank 64-bit PC (AMD64) desktop image. The script used to fully reproduce the bug on said VM, and the resulting log files can be found right below:
  • install-archive.log
  • install-rpy2.log
  • rpy2-import-archive-error.log
  • Code:

      #!/usr/bin/env bash
      set -euxo pipefail
    
      sudo apt-get update -y
    
      # ---- Install base python3, pip3 and R distribution
      sudo apt-get install -y python3-pip
      sudo apt-get install -y r-base-core r-base r-recommended
    
      # ---- Add default .local to the user's PATH
      echo 'export PATH="$HOME/.local/bin:${PATH}"' >> ~/.bashrc && . ~/.bashrc
       
      # ---- Install dependencies for r-archive
      sudo apt-get install -y libarchive-dev
    
      # ---- Pre-configure personal user library
      R --slave -e 'dir.create(path = Sys.getenv("R_LIBS_USER"), recursive = TRUE)'
      
      # ---- Install r-archive package
      R --slave -e 'install.packages("archive"); library(archive); packageVersion("archive")' |& tee > install-archive.log
      
      # ---- Install rpy2-v3.5.16
      pip3 install --no-input -U pip setuptools && pip3 install --no-input rpy2==3.5.16 |& tee install-rpy2.log
    
      # ---- Attempt to import archive from python, using rpy2
      python3 -c 'from rpy2.robjects.packages import importr; importr("archive")' |& tee rpy2-import-archive-error.log

Getting pretty stumped by this, so thanks in advance for any help in deciphering this, as the LD_PRELOAD "fix" seems dodgy at best...

Cheers!