@magnetikonline , thank you for proposing changes!

I have a question to clarify - your code will apply LD_LIBRARY_PATH value only for the next 2 commands:

./python -m ensurepip
./python -m pip install --ignore-installed pip

LD_LIBRARY_PATH variable won't be saved for next steps in your workflow. Is it expected?

If we want to pass LD_LIBRARY_PATH value to all next steps we will need something like that on task level: https://github.com/actions/setup-python/blob/main/src/find-python.ts#L111

Hey @maxim-lobanov - it's a good point. So if you look at my linked issue here: actions/setup-python#131 - you'll see it seems to only effect these initial pip steps.

So personally, we're using self-hosted runners as EC2s baked from AMI's. In those AMIs I'm bootstrapping multiple Python versions - by downloading manually the desired python-versions archives - decompressing and then running the ./setup.sh within.

So then, once booted (we only use each runner once then kill it) - a call to the setup-python action finds the required Python already in the tool cache - avoiding the download.

Right now, I only have to tweak LD_LIBRARY_PATH when running ./setup.sh (which otherwise bombs at the pip steps) - once the baked AMI is then restarted as a runner - LD_LIBRARY_PATH is unset/empty - and everything works as expected - including the use of pip.

Does that make sense?

What's the status here? I was just setting up self-hosted runners for us and ran into this issue. Unfortunately the workarounds for setting the library path posted elsewhere don't look like they will work for me as the runner setup is injecting some uuids that I can't figure out where they are coming from, so can't preemptively set the path.

@jbooth-mastery I'd love to see this merged.

Out of interest, did you try actions/setup-python#131 (comment) ? Whilst it's a clunky solution, it does prove that what I'm proposing here as a PR should do the job.

Sorry @maxim-lobanov - are you able to offer any insight if we can get this merged (and Python manifests rebuilt from it?).

That's the workaround I was referring to that doesn't work for me because I'm getting a uuid in the path that python is getting unpacked to from tar and I can't find said uuid anywhere else (env var, etc), so I can't pre-seed it into the LD_LIBRARY_PATH.

cc: @konradpabjan to review this PR

Hey @magnetikonline

First off, thanks for the PR! ❤️ Due to the nature of self-hosted runners and different settings/configurations, diagnosing and fixing issues with setup-python can be very difficult 😅

I recall hitting error while loading shared libraries: libpython3.7m.so.1.0: cannot open shared object file: No such file or directory a few times in a past. With my local machines at least, the error was always something related to where Python was getting installed and unzipped because I would forget sometimes to set some env variables locally that are taken care of in hosted env. Did you follow the instructions here and do anything to any env variables (specifically AGENT_TOOLSDIRECTORY)? https://github.com/actions/setup-python#using-setup-python-with-a-self-hosted-runner

Follow up question, from your logs in the linked issue, it looks like you got passed Create Python 3.7.5 folder. Do you know where that folder gets created? Depending on what env variables are set, it can create opt/hostedtoolscache/3.7.5 or some other folder in a different location. I'm curious if this might be the issue.

Thanks @konradpabjan for the reply.

So I'm not setting AGENT_TOOLSDIRECTORY directly - that might be the next thing I try, see if that fixes dynamic Python version installs.

What I am doing, is preloading manifests of Python into a baked AMI, so it skips the download of the manifest. This is done by:

• Setting/exporting RUNNER_TOOL_CACHE to /home/runner/_work/_tool
• exporting LD_LIBRARY_PATH as /home/runner/_work/_tool/Python/PY_VERSION/x64/lib (as per this PR).
• running the Python installer ./setup.sh
• Finally symlinking /home/runner/_work/_tool/Python/PY_VERSION over to /opt/hostedtoolcache/Python

I figured all this out by reverse engineering the GitHub cloud runners and some trial and error (from memory!).

Follow up question, from your logs in the linked issue, it looks like you got passed Create Python 3.7.5 folder. Do you know where that folder gets created? Depending on what env variables are set, it can create opt/hostedtoolscache/3.7.5 or some other folder in a different location. I'm curious if this might be the issue.

Does the above answer your question here?

Out of interest, is AGENT_TOOLSDIRECTORY specific to the Python Action, or all language installer based actions?

Thanks for the clarification @magnetikonline

I did some testing with the steps you provided and it works! A combination of the LD_LIBRARY_PATH variable and a symlink to opt/hostedtoolcache/Python does the trick and I'm able to get workflows to successfully run. I never knew about this LD_LIBRARY_PATH env variable.

I think this type of setup would be useful for lots of customers and it provides an alternative way to get setup-python up and running on a self-hosted runner compared to what we currently have documented. With regards to the changes in this PR I have no objections 👍 Can't think of any negative consequences and this should not break any existing workflows if added. Some extra documentation can be added to the README for setup-python about how this can be used. @brcrista thoughts?

AGENT_TOOLSDIRECTORY is an env variable that is set on our hosted runners and it's where certain cached Python distributions are installed out of the box. No other setup actions use this location and setup-python has to do some special things because we compile with the --enable-shared flag which makes the whole installation non-portable. Other installers can normally install to different locations but with Python there are a lot of extra hoops that we have to jump through because it's compiled in a specific location.

Not a problem @konradpabjan 👍

That would be great - I wonder, is it possible to maybe test this with a single manifest release - maybe with a beta/test tag that I can use with the actions/setup-python action so we can both confirm this end to end?

Forgot to mention yesterday - user @rmileskcl had success this way - which is pretty much what this PR will bake into the setup.sh:

actions/setup-python#131 (comment)

Thanks @brcrista & @konradpabjan

Made that suggestion in 62b0db9 - which is good, as it mirrors the Powershell installer message somewhat now:

Oh and sorry, must have been late at night - should have done this work on a branch in my repo clone 🤦 - though I don't think it should matter.

Thanks for the update @magnetikonline! The correct changes are in, in the end it doesn't matter 😃

Lets merge this in and move forward @maxim-lobanov. I propose we first update just two versions for testing (3.7.5 and 3.7.9?). One version that must always be downloaded and a second version that gets pre-installed during image generation.

Great news @konradpabjan - I'll await the merge and test this at our end. Are the manifests re-baked on merge to main - or is it a manual process from your end?

@magnetikonline , it is manual but pretty quick process.
We will take care about merging this PR and releasing new versions next week.
Sorry for delay, this week we were overloaded with Apple releases (Xcode / iOS stuff)

@maxim-lobanov no need to apologise at all! Just want to be sure I can do my bit to re-test at my end to confirm this PR does in fact do the job it's designed to so 😄

Hello @magnetikonline ! We've built Python versions (3.8.2, 3.7.7, 3.7.5, 3.6.9) with your fix and published them to the MaksimZhukov/python-versions repository in order to test. Could you please specify path to the MaksimZhukov/setup-python action in your workflow to make sure that your fix works as expected?

strategy:
  matrix:
    python: [3.6.9, 3.7.5, 3.7.7, 3.8.2]
steps:
- name: setup-python ${{ matrix.python }}
  uses: MaksimZhukov/setup-python@main
  with:
    python-version: ${{ matrix.python }}

Hey @MaksimZhukov - just tested against our self hosted runners.

The install works great - and pip installs as expected... but... (always a but!).

A following step of python --version fails with the same lib error style:

python: error while loading shared libraries: libpython3.7m.so.1.0: cannot open shared object file: No such file or directory

So.... I really need to be doing the final step of (as per my previous comment):

Finally symlinking /home/runner/_work/_tool/Python/PY_VERSION over to /opt/hostedtoolcache/Python

.... so, maybe I should just look at setting AGENT_TOOLSDIRECTORY in the Runner .env file as you've all suggested earlier. As we probably don't want to have the action itself doing symlinks all over the filesystem (probably a little too evasive)?

The other option, maybe the action could be updated to set LD_LIBRARY_PATH after install? E.g.

core.exportVariable
echo "::set-env name=LD_LIBRARY_PATH::/home/runner/_work/_tool/Python/VERSION/x64/lib"

So then with LD_LIBRARY_PATH set for the rest of the run - all should be fine? Would need a second PR for the setup-python Action itself of course.

@magnetikonline , that was I talking about in #49 (comment)
IMHO, we should set this variable on actions/setup-python level.
@MaksimZhukov can you try to PoC it in your fork please?

Yep @maxim-lobanov - fair call, my oversight 😄

But I think you'll need it set in both the manifests and then the Action (the later being for the following steps) - correct?

I've updated my fork and added the LD_LIBRARY_PATH variable. @magnetikonline could you please restart your workflow run?

Hey @MaksimZhukov - yep that works wonderfully. 👍

If I could add one thing, make the update to LD_LIBRARY_PATH additive, right now you're just setting it - https://github.com/MaksimZhukov/setup-python/commit/227460a3a5f83574a27b370cc60673d3652955df#diff-7159c6139aba803e6290bd5cb287d23fR111 that way you're not clobbering a LD_LIBRARY_PATH that's already set.

And I still think this PR needs to be included - as you can't set LD_LIBRARY_PATH within the initial action step that runs the manifest. But with both these in place, I think we're good 👍

Thank you for checking, @magnetikonline .

Actually, in this case, we can avoid patching of all python packages here. We can just apply LD_LIBRARY_PATH on action level:

1. core.exportVariable - to apply variable for next steps
2. process.env.LD_LIBRARY_PATH - to apply variable for current step and setup.sh

I would leave this decision to @konradpabjan as owner of actions/setup-python.

@konradpabjan could you please revisit this PR one more time?
After additional testing we have found that it is not enough to change setup.sh file. we also need to apply LD_LIBRARY_PATH value to next steps. So I guess it makes sense to set this variable on task level. If we set it as I described above we don't need to patch existing packages at all and it will be more convenient to have all logic in the single place.

Also a few thoughts about LD_LIBRARY_PATH, we shouldn't replace it fully for next steps because it can break some other existing logic so we should UPDATE it.

LD_LIBRARY_PATH
In Linux, the environment variable LD_LIBRARY_PATH is a colon-separated set of directories where libraries should be searched for first, before the standard set of directories

I guess we can do something like

const updatedLibraryPath = `${process.env.LD_LIBRARY_PATH}:${pythonInstallationDir}\dir`
core.exportVariable("LD_LIBRARY_PATH", updatedLibraryPath)
process.env.LD_LIBRARY_PATH = updatedLibraryPath

Sounds great @maxim-lobanov - agreed, we can do this in the action only as you can mutate process.env.LD_LIBRARY_PATH to control when setup.sh is called 👍

Also a few thoughts about LD_LIBRARY_PATH, we shouldn't replace it fully for next steps because it can break some other existing logic so we should UPDATE it.

Exactly, as I wrote above - certainly should append to LD_LIBRARY_PATH as a final version - your example above isn't quite right though - as if LD_LIBRARY_PATH is empty - you'll have a leading/floating colon - so need to check if empty before append. Or maybe it doesn't really matter....

@magnetikonline @maxim-lobanov

Thank you for testing and the deep investigation!

We can go ahead and try to implement this in actions/setup-python. From the steps described, I doesn't seem like it should be that much work, I can help review any PRs to move this forward, core.exportVariable should do the trick.

Any potential changes can first go to actions/setup-python@main and if there are no issues then the corresponding tags (such as v2) can be updated. If some issues come up, it should be fairly easy to revert but I don't expect that to happen. Implementing everything inside the action rather than the setup script makes me feel better since it will give us more flexibility.

Hello @magnetikonline ! Would you like to provide a pull request to the actions/setup-python repository? If you don't have a chance to do this, our team will take care of it in a few weeks.

@maxim-lobanov

Hello @MaksimZhukov - yeah I reckon I can give that a good. Can hopefully scrape together some free time this weekend.

cc: @maxim-lobanov

Done 👍

actions/setup-python#144

Hopefully this meets the brief @MaksimZhukov / @maxim-lobanov - quite nice to write this, haven't done any TypeScript before - think I need to dig in a little more on my next pet project - very enjoyable.

Closing this out - as we've fixed it within the action itself at: actions/setup-python@878156f