Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Mypy issue validating package with directory and -p arg #11234

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
oleksiykamenyev opened this issue Sep 30, 2021 · 2 comments · Fixed by #13733
Closed

Mypy issue validating package with directory and -p arg #11234

oleksiykamenyev opened this issue Sep 30, 2021 · 2 comments · Fixed by #13733
Labels
crash

Comments

@oleksiykamenyev
Copy link

Bug Report
I'm having an issue running mypy on a package with the -p argument where the package directory has a subdirectory inside it. I'm receiving the following error with mypy 0.9.10:

Traceback (most recent call last):
  File "/Users/oleksiyk/.pyenv/versions/3.8.9/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Users/oleksiyk/.pyenv/versions/3.8.9/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/oleksiyk/virtualenv_python38/lib/python3.8/site-packages/mypy/__main__.py", line 23, in <module>
    console_entry()
  File "mypy/build.py", line 1955, in wrap_context
  File "mypy/build.py", line 2205, in finish_passes
  File "mypy/build.py", line 818, in report_file
  File "mypy/report.py", line 83, in file
  File "mypy/report.py", line 482, in on_file
  File "mypy/report.py", line 131, in iterate_python_lines
  File "/Users/oleksiyk/.pyenv/versions/3.8.9/lib/python3.8/tokenize.py", line 392, in open
    buffer = _builtin_open(filename, 'rb')
IsADirectoryError: [Errno 21] Is a directory: 'python_tools/resources'

I tested with the master branch, and the error is the same, but the traceback is different:

Traceback (most recent call last):
  File "/Users/oleksiyk/.pyenv/versions/3.8.9/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Users/oleksiyk/.pyenv/versions/3.8.9/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/oleksiyk/virtualenv_python38/lib/python3.8/site-packages/mypy/__main__.py", line 23, in <module>
    console_entry()
  File "/Users/oleksiyk/virtualenv_python38/lib/python3.8/site-packages/mypy/__main__.py", line 11, in console_entry
    main(None, sys.stdout, sys.stderr)
  File "/Users/oleksiyk/virtualenv_python38/lib/python3.8/site-packages/mypy/main.py", line 87, in main
    res, messages, blockers = run_build(sources, options, fscache, t0, stdout, stderr)
  File "/Users/oleksiyk/virtualenv_python38/lib/python3.8/site-packages/mypy/main.py", line 165, in run_build
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File "/Users/oleksiyk/virtualenv_python38/lib/python3.8/site-packages/mypy/build.py", line 179, in build
    result = _build(
  File "/Users/oleksiyk/virtualenv_python38/lib/python3.8/site-packages/mypy/build.py", line 254, in _build
    graph = dispatch(sources, manager, stdout)
  File "/Users/oleksiyk/virtualenv_python38/lib/python3.8/site-packages/mypy/build.py", line 2707, in dispatch
    process_graph(graph, manager)
  File "/Users/oleksiyk/virtualenv_python38/lib/python3.8/site-packages/mypy/build.py", line 3031, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/Users/oleksiyk/virtualenv_python38/lib/python3.8/site-packages/mypy/build.py", line 3132, in process_stale_scc
    graph[id].finish_passes()
  File "/Users/oleksiyk/virtualenv_python38/lib/python3.8/site-packages/mypy/build.py", line 2220, in finish_passes
    free_tree(self.tree)
  File "/Users/oleksiyk/.pyenv/versions/3.8.9/lib/python3.8/contextlib.py", line 131, in __exit__
    self.gen.throw(type, value, traceback)
  File "/Users/oleksiyk/virtualenv_python38/lib/python3.8/site-packages/mypy/build.py", line 1962, in wrap_context
    yield
  File "/Users/oleksiyk/virtualenv_python38/lib/python3.8/site-packages/mypy/build.py", line 2215, in finish_passes
    manager.report_file(self.tree, self.type_map(), self.options)
  File "/Users/oleksiyk/virtualenv_python38/lib/python3.8/site-packages/mypy/build.py", line 821, in report_file
    self.reports.file(file, self.modules, type_map, options)
  File "/Users/oleksiyk/virtualenv_python38/lib/python3.8/site-packages/mypy/report.py", line 85, in file
    reporter.on_file(tree, modules, type_map, options)
  File "/Users/oleksiyk/virtualenv_python38/lib/python3.8/site-packages/mypy/report.py", line 483, in on_file
    for lineno, line_text in iterate_python_lines(path):
  File "/Users/oleksiyk/virtualenv_python38/lib/python3.8/site-packages/mypy/report.py", line 133, in iterate_python_lines
    with tokenize.open(path) as input_file:
  File "/Users/oleksiyk/.pyenv/versions/3.8.9/lib/python3.8/tokenize.py", line 392, in open
    buffer = _builtin_open(filename, 'rb')
IsADirectoryError: [Errno 21] Is a directory: 'python_tools/resources'

Without the -p argument, the command passes with no issue.

Expected Behavior

I'm not too familiar with mypy, but I would expect the code to ignore such directories I think. I'm also not sure why the -p argument would cause this issue, when the code otherwise works normally. When I dug through the traceback a bit, I found this function

mypy/mypy/report.py

Lines 121 to 128 in f98f782

def should_skip_path(path: str) -> bool:
if stats.is_special_module(path):
return True
if path.startswith('..'):
return True
if 'stubs' in path.split('/') or 'stubs' in path.split(os.sep):
return True
return False
that seems to handle skipping different paths for mypy validation. For the sake of testing, I changed the directory name above from resources to stubs, and that fixed the error. Perhaps all directories should be ignored by this skip function (or maybe they shouldn't be passed into it in the first place, not sure)?

My Environment

  • Mypy version used: 0.9.10 and 0.920+dev.209a7193feb4bbfa38d09232b0a5a916e9d2e605
  • Full command run: python3.8 -m mypy --txt-report ~/tmp/mypyr_report --install-types --non-interactive --namespace-packages --ignore-missing-imports -p yahoo.contrib.mx3_python_tools --show-traceback
  • Mypy configuration options from mypy.ini (and other config files): none
  • Python version used: 3.8.9 (although in a separate env, 3.9 showed the same issue)
  • Operating system and version: MacOS Big Sur version 11.6
@oleksiykamenyev oleksiykamenyev added the bug mypy got something wrong label Sep 30, 2021
@jamesbraza
Copy link
Contributor

I am getting a similar crash when using something like mypy -p pkg --html-report out with mypy==0.920 on a native namespace package.

Seems to be a bug in reporting related to -p

Traceback (most recent call last):
  File "/Users/james.braza/code/pkg/venv/bin/mypy", line 8, in <module>
    sys.exit(console_entry())
  File "mypy/build.py", line 1962, in wrap_context
  File "mypy/build.py", line 2215, in finish_passes
  File "mypy/build.py", line 821, in report_file
  File "mypy/report.py", line 85, in file
  File "mypy/report.py", line 483, in on_file
  File "mypy/report.py", line 133, in iterate_python_lines
  File "/Users/james.braza/.pyenv/versions/3.8.12/lib/python3.8/tokenize.py", line 392, in open
    buffer = _builtin_open(filename, 'rb')
IsADirectoryError: [Errno 21] Is a directory: 'pkg/path/to/dir'

@zevisert
Copy link

Also seeing this on 0.940+dev.14de8c6f1f24648299528881561ddcd52172701d.

I am blocked from using any reports, since my project is a namespace project that has found twice under different module names.

# mypy project --cobertura-xml-report mypy-reports --show-traceback
project/db/connect.py: error: Source file found twice under different module names: "connect" and "project.db.connect"
Found 1 error in 1 file (errors prevented further checking)

Traceback:

# mypy -p project --cobertura-xml-report mypy-reports --show-traceback 

/code/project/db: error: INTERNAL ERROR -- Please try using mypy master on Github:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 0.940+dev.14de8c6f1f24648299528881561ddcd52172701d
Traceback (most recent call last):
  File "/opt/tooling/pyenv/versions/hula-env/bin/mypy", line 8, in <module>
    sys.exit(console_entry())
  File "/opt/tooling/pyenv/versions/3.10.1/envs/project-env/lib/python3.10/site-packages/mypy/__main__.py", line 12, in console_entry
    main(None, sys.stdout, sys.stderr)
  File "/opt/tooling/pyenv/versions/3.10.1/envs/project-env/lib/python3.10/site-packages/mypy/main.py", line 96, in main
    res, messages, blockers = run_build(sources, options, fscache, t0, stdout, stderr)
  File "/opt/tooling/pyenv/versions/3.10.1/envs/project-env/lib/python3.10/site-packages/mypy/main.py", line 173, in run_build
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File "/opt/tooling/pyenv/versions/3.10.1/envs/project-env/lib/python3.10/site-packages/mypy/build.py", line 180, in build
    result = _build(
  File "/opt/tooling/pyenv/versions/3.10.1/envs/project-env/lib/python3.10/site-packages/mypy/build.py", line 256, in _build
    graph = dispatch(sources, manager, stdout)
  File "/opt/tooling/pyenv/versions/3.10.1/envs/project-env/lib/python3.10/site-packages/mypy/build.py", line 2724, in dispatch
    process_graph(graph, manager)
  File "/opt/tooling/pyenv/versions/3.10.1/envs/project-env/lib/python3.10/site-packages/mypy/build.py", line 3068, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/opt/tooling/pyenv/versions/3.10.1/envs/project-env/lib/python3.10/site-packages/mypy/build.py", line 3169, in process_stale_scc
    graph[id].finish_passes()
  File "/opt/tooling/pyenv/versions/3.10.1/envs/project-env/lib/python3.10/site-packages/mypy/build.py", line 2209, in finish_passes
    with self.wrap_context():
  File "/opt/tooling/pyenv/versions/3.10.1/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/opt/tooling/pyenv/versions/3.10.1/envs/project-env/lib/python3.10/site-packages/mypy/build.py", line 1970, in wrap_context
    yield
  File "/opt/tooling/pyenv/versions/3.10.1/envs/project-env/lib/python3.10/site-packages/mypy/build.py", line 2225, in finish_passes
    manager.report_file(self.tree, self.type_map(), self.options)
  File "/opt/tooling/pyenv/versions/3.10.1/envs/project-env/lib/python3.10/site-packages/mypy/build.py", line 822, in report_file
    self.reports.file(file, self.modules, type_map, options)
  File "/opt/tooling/pyenv/versions/3.10.1/envs/project-env/lib/python3.10/site-packages/mypy/report.py", line 85, in file
    reporter.on_file(tree, modules, type_map, options)
  File "/opt/tooling/pyenv/versions/3.10.1/envs/project-env/lib/python3.10/site-packages/mypy/report.py", line 610, in on_file
    with tokenize.open(path) as input_file:
  File "/opt/tooling/pyenv/versions/3.10.1/lib/python3.10/tokenize.py", line 394, in open
    buffer = _builtin_open(filename, 'rb')
IsADirectoryError: [Errno 21] Is a directory: 'project/db'
/code/project/db: : note: use --pdb to drop into pdb
Generated Cobertura report: /code/project/mypy-reports/cobertura.xml

@AlexWaygood AlexWaygood added crash and removed bug mypy got something wrong labels Mar 26, 2022
@pawamoy pawamoy mentioned this issue Jul 20, 2022
Closed
hauntsaninja added a commit to hauntsaninja/mypy that referenced this issue Sep 26, 2022
@hauntsaninja
Fix crash with report generation on namespace packages
58a4e1d 
Fixes python#11234
@hauntsaninja hauntsaninja mentioned this issue Sep 26, 2022
Merged
hauntsaninja added a commit that referenced this issue Sep 27, 2022
@hauntsaninja
Fix crash with report generation on namespace packages (#13733)
ab3b98f 
Fixes #11234
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
crash
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix crash with report generation on namespace packages hauntsaninja/mypy
4 participants
@jamesbraza @oleksiykamenyev @zevisert @AlexWaygood