.pth file has no effect in editable install #43

chrisjbillington · 2020-05-22T22:57:41Z

.pth files only work if they're in a site packages directory. Editable installs add the current directory to the path, but they don't add them as site directories so no .pth files get processed.

Perhaps labscript_utils should process the .pth file at import time, if it sees that it has not already been processed. Then user code will be available in the interpreter so long as a labscript suite module has been imported.

chrisjbillington · 2020-06-01T14:13:30Z

Extra comments from a duplicate issue I filed after forgetting I already filed this one:

This .pth file adds userlib and pythonlib (as defined in labconfig) to the python import path to make user code available for import from the python interpreter regardless of whether the code is running within a labscript suite program.

We could:

Work out how to get it into a site directory during an editable install (possibly not reversible or otherwise a bit magic)
Convince Python that .egg-link files should imply the target directory is a site directory to be processed by the site module, make a patch and wait for them to incorporate it
Work out if there's some way to otherwise add our package directory as a site directory during editable install.
Live with it and instead process the .pth file whenever labscript_utils is imported. This means user code would not be available for import in editable installs when not running inside a labscript suite program, unless labscript_utils has been imported. This could simply be documented as a limitation of editable installs.

rpanderson · 2020-06-23T22:27:41Z

Live with it and instead process the .pth file whenever labscript_utils is imported.

I'm fine with this less magical option that will be easily documented and implemented by those with an editable install.

philipstarkey · 2020-06-24T00:43:14Z

Work out how to get it into a site directory during an editable install (possibly not reversible or otherwise a bit magic)

Can we not just modify labscript-profile-create to locate the site-packages directory using site.getsitepackages() and then copy the .pth file there if it doesn't already exist?

chrisjbillington · 2020-06-24T01:18:56Z

Can we not just modify labscript-profile-create to locate the site-packages directory using site.getsitepackages() and then copy the .pth file there if it doesn't already exist?

Absolutely. Just that this isn't reversible - if the user subsequently uninstalls the labscript suite modules, the .pth file will cause import errors to be printed every time they start a Python interpreter. Perhaps if I wrap its code in try:except: it will be harmless to leave the .pth file behind. But the code you can write in a .pth file is very limited, so even a try:except: might not be possible! I'll see. If so that is a decent solution IMHO.

edit:

using site.getsitepackages()

It's not that simple unfortunately, since you can't know whether the user site package or regular site packages is the right place. But I know what hoops to jump through to figure out which site directory is associated with an editable install. That logic is in some desktop-app functions where it has to figure out where the appropriate scripts directory is for an editable install (otherwise pretty tricky to work out!)

philipstarkey · 2020-06-24T01:36:03Z

Can we not just modify labscript-profile-create to locate the site-packages directory using site.getsitepackages() and then copy the .pth file there if it doesn't already exist?

Absolutely. Just that this isn't reversible - if the user subsequently uninstalls the labscript suite modules, the .pth file will cause import errors to be printed every time they start a Python interpreter. Perhaps if I wrap its code in try:except: it will be harmless to leave the .pth file behind. But the code you can write in a .pth file is very limited, so even a try:except: might not be possible! I'll see. If so that is a decent solution IMHO.

It's possible, but messy. You need to make the .pth file something like:

import sys; exec("try: import labscript_profile; labscript_profile.add_userlib_and_pythonlib()\nexcept ModuleNotFoundError: pass")
import sys; exec("try: import labscript_profile; labscript_profile.add_development_directories()\nexcept ModuleNotFoundError: pass")

This is because only lines starting with import are only ones that are executed (see here) so it has to start with an import that will succeed, but this can be followed by another arbitrary exec. Very much a hack, but then I'd argue the existing .pth file was just as much a hack to execute a specific function in labscript_utils.

edit:

using site.getsitepackages()

It's not that simple unfortunately, since you can't know whether the user site package or regular site packages is the right place. But I know what hoops to jump through to figure out which site directory is associated with an editable install. That logic is in some desktop-app functions where it has to figure out where the appropriate scripts directory is for an editable install (otherwise pretty tricky to work out!)

Sure!

philipstarkey · 2020-06-24T01:53:51Z

So the directory to put the .pth file in should be able to be found using desktop_app.environment._get_install_directory('labscript_utils')?

chrisjbillington · 2020-06-24T02:36:56Z

Correct!

Fixes labscript-suite#43 The .pth file is now copied during `labscript-profile-create` if it does not exist in the site-packages dir (where the .egg-info files exist) and if it the .pth file exists in the package structure (aka labscript-utils it is an editable install) The labscript-suite.pth file has been updated to not raise an exception if labscript-suite has been uninstalled (so we do not have to worry about cleanup). Also updated the `labscript_profile.add_userlib_and_pythonlib` function to use the `site` library for adding userlib and pythonlib to the system path.

commit d8b8343 Merge: 9bc7a39 bfa6932 Author: Chris Billington <[email protected]> Date: Wed Jun 24 20:48:39 2020 -0400 Merge pull request #59 from philipstarkey/fix-h5py-file-mode Addresses #47 for this module commit 9bc7a39 Merge: 1189334 66d0be4 Author: Chris Billington <[email protected]> Date: Wed Jun 24 20:30:42 2020 -0400 Merge pull request #55 from rpanderson/example_IMAQdx_remote Example labscript and analysis scripts for IMAQdxCamera and RemoteBLACS commit bfa6932 Author: philipstarkey <[email protected]> Date: Thu Jun 25 10:25:49 2020 +1000 Addresses #47 for this module commit 1189334 Merge: 5025b1b 0b0d07a Author: Chris Billington <[email protected]> Date: Wed Jun 24 19:35:15 2020 -0400 Merge pull request #54 from rpanderson/example_experiment Minimal working connection table and example experiment script commit 66d0be4 Author: Russell Anderson <[email protected]> Date: Mon Jun 15 13:37:23 2020 +1000 lyse analysis script demonstrating some Run methods for example_IMAQdx_remote commit a48a9c0 Author: Russell Anderson <[email protected]> Date: Mon Jun 15 13:15:48 2020 +1000 Example connection table and labscript experiment script for RemoteBLACS and IMAQdxCamera commit 5025b1b Merge: f77525f ed76901 Author: Chris Billington <[email protected]> Date: Wed Jun 24 02:16:29 2020 -0400 Merge pull request #58 from philipstarkey/philipstarkey/issue43 Fixes .pth file has no effect in editable install commit ed76901 Author: chrisjbillington <[email protected]> Date: Wed Jun 24 01:23:44 2020 -0400 Move copying of .pth file into setup.py Use a custom develop command to copy the .pth file into site-packages when `python setup.py develop` is run - this is what pip does under the hood for `pip install -e`. No longer need desktop-app to locate site-packages, as the setuptools command class knows it as `self.install_dir`. The file is not removed by `pip uninstall`, but due to the `try: except:`s in the `.pth` file, leaving it behind is harmless. It is removed if the user runs whatever `setup.py` command undoies commit 894e245 Author: chrisjbillington <[email protected]> Date: Wed Jun 24 00:04:13 2020 -0400 Require desktop-app 0.2.6 Which renames `_get_install_directory` to not have an underscore, and which fixes a bug preventing docs from building on RTD. commit 271439e Author: philipstarkey <[email protected]> Date: Wed Jun 24 12:53:49 2020 +1000 Removed support for custom development directories from path commit dd098a5 Author: philipstarkey <[email protected]> Date: Wed Jun 24 12:38:19 2020 +1000 Fixes .pth file has no effect in editable install Fixes #43 The .pth file is now copied during `labscript-profile-create` if it does not exist in the site-packages dir (where the .egg-info files exist) and if it the .pth file exists in the package structure (aka labscript-utils it is an editable install) The labscript-suite.pth file has been updated to not raise an exception if labscript-suite has been uninstalled (so we do not have to worry about cleanup). Also updated the `labscript_profile.add_userlib_and_pythonlib` function to use the `site` library for adding userlib and pythonlib to the system path. commit 0b0d07a Author: Russell Anderson <[email protected]> Date: Mon Jun 22 17:48:00 2020 +1000 start()/stop() in __main__ block so connection table can be imported commit 4c19de9 Author: Russell Anderson <[email protected]> Date: Mon Jun 15 13:12:52 2020 +1000 Add AnalogOut with ramp example, rename DigitalOut connection commit 9b64a0d Author: Russell Anderson <[email protected]> Date: Wed Jun 10 23:09:03 2020 +1000 Minimal example connection table and labscript using dummy devices commit f77525f Author: Phil Starkey <[email protected]> Date: Sat Jun 20 18:09:53 2020 +1000 Removed unnecessary fixed title from main TOC This was erroneously added in #57 commit 5e02b01 Merge: 3f78207 8b19766 Author: Phil Starkey <[email protected]> Date: Sat Jun 20 18:07:04 2020 +1000 Merge pull request #57 from philipstarkey/master Changed URL for API reference to be shorter Fixed bug with readthedocs auto PR build commit 8b19766 Author: philipstarkey <[email protected]> Date: Sat Jun 20 12:02:51 2020 +1000 Updated method call for stylesheet to remove deprecated warning in build commit 56fdd63 Author: philipstarkey <[email protected]> Date: Sat Jun 20 11:46:33 2020 +1000 Fixed sphinx conf to detect PR builds and link to 'latest' instead of stable commit 9889d19 Author: philipstarkey <[email protected]> Date: Sat Jun 20 10:44:04 2020 +1000 Changed URL for API reference to be shorter commit 3f78207 Merge: bbd928a bf9d10e Author: Phil Starkey <[email protected]> Date: Fri Jun 19 18:58:37 2020 +1000 Merge pull request #56 from philipstarkey/master Added links to other component docs commit bf9d10e Author: philipstarkey <[email protected]> Date: Fri Jun 19 16:06:09 2020 +1000 Addressing latest review comments commit e75b6bc Author: philipstarkey <[email protected]> Date: Fri Jun 19 11:23:49 2020 +1000 Updated docs to use jinja template for component doc links commit 9c61238 Author: philipstarkey <[email protected]> Date: Thu Jun 18 17:14:57 2020 +1000 Removed autogenerated rst file from version control commit 2303185 Author: philipstarkey <[email protected]> Date: Thu Jun 18 16:43:29 2020 +1000 Updated .gitignore with latest GitHub defaults commit 87b757e Author: philipstarkey <[email protected]> Date: Thu Jun 18 16:39:52 2020 +1000 Relocated custom sphinx gitignore rules commit 75813a5 Author: philipstarkey <[email protected]> Date: Thu Jun 18 16:31:01 2020 +1000 Updated PR as per review in labscript-suite/labscript-suite#48 commit bbd928a Author: Russell Anderson <[email protected]> Date: Wed Jun 17 11:39:17 2020 +1000 Populated README.md with styling, iconogrpahy, prose, and badges commit 573c521 Author: philipstarkey <[email protected]> Date: Tue Jun 16 18:10:25 2020 +1000 Added links to other component docs

rpanderson mentioned this issue May 23, 2020

Remove code specific to building with setuptools_conda #41

Merged

philipstarkey mentioned this issue Jun 1, 2020

.pth file not processed in editable install #50

Closed

philipstarkey self-assigned this Jun 24, 2020

philipstarkey mentioned this issue Jun 24, 2020

Fixes .pth file has no effect in editable install #58

Merged

chrisjbillington closed this as completed in #58 Jun 24, 2020

zakv mentioned this issue Jul 15, 2020

Can't Imort userlib #63

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

.pth file has no effect in editable install #43

.pth file has no effect in editable install #43

chrisjbillington commented May 22, 2020

chrisjbillington commented Jun 1, 2020

rpanderson commented Jun 23, 2020

philipstarkey commented Jun 24, 2020

chrisjbillington commented Jun 24, 2020 •

edited

Loading

philipstarkey commented Jun 24, 2020

philipstarkey commented Jun 24, 2020

chrisjbillington commented Jun 24, 2020

.pth file has no effect in editable install #43

.pth file has no effect in editable install #43

Comments

chrisjbillington commented May 22, 2020

chrisjbillington commented Jun 1, 2020

rpanderson commented Jun 23, 2020

philipstarkey commented Jun 24, 2020

chrisjbillington commented Jun 24, 2020 • edited Loading

philipstarkey commented Jun 24, 2020

philipstarkey commented Jun 24, 2020

chrisjbillington commented Jun 24, 2020

chrisjbillington commented Jun 24, 2020 •

edited

Loading