In order for a python project to be importable when it consists of both .py files and compiled extensions, you have to do one of:

• have them be different top-level importable modules, no shared namespace

They are separate importable modules. However, the pure-Python module cannot find the C extension. This is the source tree after the mason-python build:

.
+- build/
|  +- cp311/
|     +- ...
|     +- src/
|        +- ext/
|           +- myext.cpython-311-x86_64-linux-gnu.so
+- src
|  +- ext
|     +- myext.c
+- module
|  +- __init__.py
+- main.py

main.py imports module needs to import myext. However, module can't find myext even with PYTHONPATH set.

• install the project (at least in editable mode) and merge the source and build trees together

I am not sure what this means. I am using editable builds but do you mean that after the build I would have to copy all of the pure-Python files into the build directory build/cp311?

If PYTHONPATH contains both:

• $PWD (to execute the statement import module)
• $PWD/build/cp11/src/ext (to execute the statement import myext)

Then in principle I can't see what the problem would be when running directly from the source tree. Have you tried adding e.g. a debug print to main.py to have it show what it sees as sys.path when the failure occurs?

I think in this example it would work, but if you modify it slightly to:

• src/mymodule/__init__.py
• src/mymodule/ext.c

then the issue is that you wind up with two conflicting locations for the same module:

• src/mymodule (contains the py files)
• build/src/mymodule (contains the shared libraries)

You can add both to your PYTHONPATH, but python stops searching once it thinks it knows where the module lives. So, for example,

from mymodule.ext import foo

will fail because python sees src/mymodule/__init__.py and decides that "mymodule" lives at src/mymodule.

If I drop the src location from PYTHONPATH, I can import the extension, and if I drop the build location, I can import __init__.py, but getting both to work at the same time is an open problem.

@eli-schwartz Does it means that we cannot run test on a wheel (because i don't think it is possible to install a wheel in editable)?

Not completely sure what you mean by that. "Editable installs" per the specification is just a wheel that doesn't contain your code, but instead contains a small import hook that locates your code from your build tree.

It's possible to test a non-editable wheel too. But your tests have to be separately importable from outside the wheel. For example, you can have

myproject/
tests/

Or you can have

src/myproject/
tests/

But what you cannot do is:

src/myproject/
src/myproject/tests/

(which is a bad idea anyway), because the tests cannot be imported in the same python process as the project itself.