Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Namespace package __path__ manipulation breaks packages using __init__.py to define code #1464

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
dzbarsky opened this issue Oct 5, 2023 · 6 comments
Closed

Namespace package __path__ manipulation breaks packages using __init__.py to define code #1464

dzbarsky opened this issue Oct 5, 2023 · 6 comments
Labels
type: pip pip/pypi integration

Comments

@dzbarsky
Copy link
Contributor

dzbarsky commented Oct 5, 2023

🐞 bug report

Affected Rule

https://github.com/bazelbuild/rules_python/blob/fe33a4582c37499f3caeb49a07a78fc7948a8949/python/pip_install/tools/wheel_installer/namespace_pkgs.py#L90-L99

Is this a regression?

Not sure, first time I have tried this.

Description

The code in the package's source init.py seems to get overwritten. I'm not fully sure why, it appears that https://github.com/bazelbuild/rules_python/blob/fe33a4582c37499f3caeb49a07a78fc7948a8949/python/pip_install/tools/wheel_installer/namespace_pkgs.py#L87-L88 is trying to guard against it.

If possible, it would be nice to lay out a site_packages that is less hacky.

🔬 Minimal Reproduction

https://github.com/dzbarsky/rules_python_pylink_bug

bazel test //:flink_import_test

🔥 Exception or Error


ImportError: cannot import name 'add_version_doc' from 'pyflink' (/private/var/tmp/_bazel_zbarsky/45e7e567ddccd93ad1b41c75488b912d/sandbox/darwin-sandbox/13/execroot/__main__/bazel-out/darwin-fastbuild/bin/flink_import_test.runfiles/pip_apache_flink_libraries/site-packages/pyflink/__init__.py)

🌍 Your Environment

Operating System:

Output of bazel version:

  
Bazelisk version: development
Build label: 6.3.2
Build target: bazel-out/darwin-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Tue Aug 8 15:58:06 2023 (1691510286)
Build timestamp: 1691510286
Build timestamp as int: 1691510286

Rules_python version:

  
0.25

Anything else relevant?
@github-actions GitHub Actions
Copy link

github-actions bot commented Apr 2, 2024

This issue has been automatically marked as stale because it has not had any activity for 180 days. It will be closed if no further activity occurs in 30 days.
Collaborators can add an assignee to keep this open indefinitely. Thanks for your contributions to rules_python!

@github-actions github-actions bot added the Can Close? Will close in 30 days if there is no new activity label Apr 2, 2024
@dzbarsky
Copy link
Contributor Author

This is still an active issues, please keep open

@aignas aignas added type: bug type: pip pip/pypi integration and removed Can Close? Will close in 30 days if there is no new activity labels Apr 20, 2024
@groodt
Copy link
Collaborator

groodt commented Aug 24, 2024

Potentially related: #2156

I'm not completely sure that this is caused by lack of site-packages, so I won't close this as a duplicate.

@groodt groodt closed this as completed Aug 24, 2024
@groodt groodt reopened this Aug 24, 2024
@engnatha
Copy link

I was able to get around this by setting enable_implicit_namespace_pkgs = True in the pip_parse target.

@engnatha
Copy link

engnatha commented Dec 3, 2024

To follow up on my comment above, while this does allow the import to work, it wipes any data directories that don't have any python files. There's some flink directories that contain only java archive files. Still discovering when precisely those are needed.

@aignas aignas removed the type: bug label Mar 21, 2025
@groodt
Copy link
Collaborator

groodt commented Apr 9, 2025 •
edited
Loading

I've got to the bottom of the issue here. The issue here is indeed solved (or handled better) by #2156 Therefore, closing as duplicate.

Root cause:

  • apache-flink-libraries and apache-flink are not correctly setup as namespace packages (they are neither pkgutil or implicit namespace packages)
  • apache-flink==1.17.1 includes pyflink/__init__.py which means that it isn't an implicit namespace package. It's also not a pkgutil namespace package due to the contents. See: https://pypi-browser.org/package/apache-flink/apache_flink-1.17.1-cp39-cp39-manylinux1_x86_64.whl
  • apache-flink-libraries==1.17.1 also shares a folder called pyflink, but it doesn't include a __init__.py so it is a namespace package which shadows the pyflink namespace when included on PYTHONPATH

This only works by accident outside bazel, because pip and others all just unzip the files into the same site-packages folder 😂

Suggestions here would be:

Further references:

@groodt groodt closed this as completed Apr 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type: pip pip/pypi integration
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@aignas @groodt @dzbarsky @engnatha