I don't know enough about the Unix build process to know if using --prefix instead of --root is the right thing to do here. I'm going to mark my review as resolved (and approve the PR) but I'd recommend getting someone who knows how Unix/MacOS installs work to check this part of the change before merging (unless, of course, you are such a person and you're happy it's right 🙂)

I can test it on *nix and macOS, but I know very little about pip/ensurepip :)

OK. Looking at the pip source, --prefix corresponds roughly to the sysconfig scheme posix_prefix. And --root can be used with --prefix, it basically rebases the installation directories on the root directory.

So if you have prefix=/foo/bar and root=/bax/quux, then you'd get purelib pointing to /baz/quux/foo/bar/lib/pythonX.Y/site-packages. With just prefix you'd get /foo/bar/lib/pythonX.Y/site-packages. And with just root you'd get /baz/quux/lib/pythonX.Y/site-packages.

There's all sorts of qualifications and special cases (because this is packaging 🙁) but that's the basic idea.

Given that the original issue was around cross-compilation for Android, I have zero idea how any of this would impact that actual scenario, though...

I would guess you'd use --root for the directory where your target rootfs is being populated, and then you'd use --prefix for the path within the target rootfs.

cc. @gpshead, since you dabble with embedded systems IIRC.

i'm not the right person to answer this either, i've poked #build on discord to see if anyone else is.

I can confirm that using --prefix for the expected location in the destination filesystem and --root(=$DESTDIR) for the current location of the virtual filesystem where the package is being prepared makes total sense. (and is how setup.py works )

Having only one or the other leads to headaches;

TL;DR: We expect --prefix to change the install location of files and the path they use to reference one another while --root should change the location we put files without changing how they refer to one another

With a basic build process and environment as such :

SOURCE_DIR=/tmp/py3.12.3
TARGET_ROOT=/tmp/build.out
TARGET_PREFIX=/opt/python/py3.12.3
cd ${SOURCE_DIR}
configure --prefix=${TARGET_PREFIX} --with-ensurepip=no [...]
make
make DESTDIR=${TARGET_ROOT} install altinstall

We get these results :

${TARGET_ROOT}${TARGET_PREFIX}/bin/python3 -m ensurepip
head -n 1 ${TARGET_ROOT}${TARGET_PREFIX}/bin/pip
#!/tmp/build.out/opt/python/py3.12.3/bin/python3

${TARGET_ROOT}${TARGET_PREFIX}/bin/python -m ensurepip --root ${TARGET_ROOT}
head -n 1 ${TARGET_ROOT}${TARGET_ROOT}${TARGET_PREFIX}/bin/pip
#!/tmp/build.out/tmp/build.out/opt/python/py3.12.3/bin/python3

In both cases the script installed contains the ${TARGET_ROOT} in the shebang when it shouldn't, but specificaly using --root behaves the way I would expect --prefix to work if no prefix had been set in the configure script.

The expected install locations for pip3 should be:

• --prefix : ${PREFIX}/bin/pip3 with shebang #!${PREFIX}/bin/python3
• --root (assuming no prefix) : ${ROOT}/bin/pip3 with shebang #!/bin/python3
• --prefix and --root : ${ROOT}${PREFIX}/bin/pip3 with shebang #!${PREFIX}/bin/python3

The behaviour becomes even weirder when using --with-ensurepip=(install|upgrade) in configure.
It triggers a call to something equivalent to :

LD_LIBRARY_PATH=${SOURCE_DIR} ${SOURCE_DIR}/python -E -m ensurepip [--upgrade] --root=${DESTDIR}

But this call ignores the --root parameter completely and just acts on the host system if an installation using the same prefix exists. The -E parameter makes setting PYTHONHOME irrelevant but omitting it doesn't resolve the issue.

With thing the way they are currently, with-ensurepip=no is the only option making sense in configure. After that, I see two quick and dirty solutions :

• calling ${TARGET_ROOT}${TARGET_PREFIX}/bin/python3 -m ensurepip on the build host, and fixing the shebang in the scripts afterwards
• adding ${TARGET_PREFIX}/bin/python3 -m ensurepip as a post-install action on the destination host

I agree that the proper solution would be to have the makefile call something along the lines of:

I'm not sure the trailing / is necessary.

One issue remains when cross-compiling (eg. AARCH64 binaries on a X86_64 host), the build host (x86_64) must be able to execute the target's binaries (AARCH64) for the actual call to work. Assuming ensurepip is pure python, the call should use the host's python3 binary (x86_64) with --prefix and --root rather than the one being built (AARCH64).

I don't see it as a blocking issue since the person building the package should be able to tell if they can use --with-ensurepip or if they should rather call ensurepip from the build host's python rather than letting the Makefile guess/try.

EDIT:
I forgot to mention that the actual module installation path follows the behaviour shown with the scripts and shebang. I just used them to illustrate the expected and actual behaviours

EDIT2: Added TL;DR an some clarifications