Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Crash when typechecking transformers 4.10.0 #11045

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
davidhewitt opened this issue Sep 2, 2021 · 3 comments · Fixed by #12023
Closed

Crash when typechecking transformers 4.10.0 #11045

davidhewitt opened this issue Sep 2, 2021 · 3 comments · Fixed by #12023
Labels
crash

Comments

@davidhewitt
Copy link

davidhewitt commented Sep 2, 2021 •
edited
Loading

Crash Report

I am typechecking a codebase which imports transformers 4.10.0 and hitting a mypy internal error. Also reported in the transformers repo as huggingface/transformers#13390

Traceback

This traceback is generated using the current mypy master (pip install git+https://github.com/python/mypy).

src/transformers/trainer.py:1435: error: INTERNAL ERROR -- Please try using mypy master on Github:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 0.920+dev.5e73cd7b0dc9b871716acf0086b2ca0376fcff39
Traceback (most recent call last):
  File "/home/david/dev/transformers/.env/bin/mypy", line 8, in <module>
    sys.exit(console_entry())
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/__main__.py", line 11, in console_entry
    main(None, sys.stdout, sys.stderr)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/main.py", line 87, in main
    res, messages, blockers = run_build(sources, options, fscache, t0, stdout, stderr)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/main.py", line 165, in run_build
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/build.py", line 179, in build
    result = _build(
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/build.py", line 254, in _build
    graph = dispatch(sources, manager, stdout)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/build.py", line 2707, in dispatch
    process_graph(graph, manager)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/build.py", line 3031, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/build.py", line 3138, in process_stale_scc
    if not graph[id].type_check_second_pass():
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/build.py", line 2192, in type_check_second_pass
    return self.type_checker().check_second_pass()
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checker.py", line 354, in check_second_pass
    self.check_partial(node)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checker.py", line 366, in check_partial
    self.accept(node)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checker.py", line 411, in accept
    stmt.accept(self)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/nodes.py", line 696, in accept
    return visitor.visit_func_def(self)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checker.py", line 743, in visit_func_def
    self._visit_func_def(defn)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checker.py", line 747, in _visit_func_def
    self.check_func_item(defn, name=defn.name)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checker.py", line 809, in check_func_item
    self.check_func_def(defn, typ, name)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checker.py", line 992, in check_func_def
    self.accept(item.body)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checker.py", line 411, in accept
    stmt.accept(self)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/nodes.py", line 1024, in accept
    return visitor.visit_block(self)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checker.py", line 2005, in visit_block
    self.accept(s)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checker.py", line 411, in accept
    stmt.accept(self)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/nodes.py", line 1216, in accept
    return visitor.visit_if_stmt(self)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checker.py", line 3350, in visit_if_stmt
    self.accept(b)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checker.py", line 411, in accept
    stmt.accept(self)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/nodes.py", line 1024, in accept
    return visitor.visit_block(self)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checker.py", line 2005, in visit_block
    self.accept(s)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checker.py", line 411, in accept
    stmt.accept(self)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/nodes.py", line 1083, in accept
    return visitor.visit_assignment_stmt(self)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checker.py", line 2047, in visit_assignment_stmt
    self.check_assignment(s.lvalues[-1], s.rvalue, s.type is None, s.new_syntax)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checker.py", line 2226, in check_assignment
    self.check_indexed_assignment(index_lvalue, rvalue, lvalue)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checker.py", line 3208, in check_indexed_assignment
    self.expr_checker.check_method_call(
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checkexpr.py", line 2457, in check_method_call
    return self.check_call(method_type, args, arg_kinds,
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checkexpr.py", line 933, in check_call
    return self.check_callable_call(callee, args, arg_kinds, context, arg_names,
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checkexpr.py", line 1024, in check_callable_call
    arg_types = self.infer_arg_types_in_context(
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checkexpr.py", line 1123, in infer_arg_types_in_context
    res[ai] = self.accept(args[ai], callee.arg_types[i])
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checkexpr.py", line 3912, in accept
    typ = node.accept(self)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/nodes.py", line 1600, in accept
    return visitor.visit_call_expr(self)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checkexpr.py", line 277, in visit_call_expr
    return self.visit_call_expr_inner(e, allow_none_return=allow_none_return)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checkexpr.py", line 326, in visit_call_expr_inner
    callee_type = get_proper_type(self.accept(e.callee, type_context, always_allow_any=True))
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checkexpr.py", line 3912, in accept
    typ = node.accept(self)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/nodes.py", line 1520, in accept
    return visitor.visit_member_expr(self)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checkexpr.py", line 2047, in visit_member_expr
    result = self.analyze_ordinary_member_access(e, is_lvalue)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checkexpr.py", line 2065, in analyze_ordinary_member_access
    member_type = analyze_member_access(
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checkmember.py", line 126, in analyze_member_access
    result = _analyze_member_access(name, typ, mx, override_info)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checkmember.py", line 143, in _analyze_member_access
    return analyze_instance_member_access(name, typ, mx, override_info)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checkmember.py", line 219, in analyze_instance_member_access
    typ = map_instance_to_supertype(typ, method.info)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/maptype.py", line 20, in map_instance_to_supertype
    if not superclass.type_vars:
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/nodes.py", line 2656, in __getattribute__
    raise AssertionError(object.__getattribute__(self, 'msg'))
AssertionError: FuncBase for non-methods lack info

To Reproduce

I can reproduce this locally in the following way:

  1. Clone the transformers repo, checkout the v4.10.0 tag
  2. Create a blank virtualenv and install mypy 0.910 (or mypy master)
  3. Add the following mypy.ini file: 
    [mypy]
check_untyped_defs=True
  4. Run mypy src/transformers/trainer.py

I'm afraid I tried to reduce to a minimal repro with no success.

Your Environment

  • Mypy version used: 0.910 and master
  • Mypy command-line flags: None other than --show-traceback
  • Mypy configuration options from mypy.ini (and other config files): See repo above (just check_untyped_defs=True)
  • Python version used: tested on 3.8.11 and 3.9.5
  • Operating system and version: Ubuntu, tested on 20.04 and 21.04
@shirayu shirayu mentioned this issue Sep 13, 2021
Closed
@shirayu
Copy link

shirayu commented Sep 29, 2021 •
edited
Loading

This may be related to #7045.

https://github.com/huggingface/transformers/blob/b91e65afe0f467e24183928bf57d92b2cef4b69f/src/transformers/trainer.py#L1435

logs["learning_rate"] = self._get_learning_rate()

calls

https://github.com/huggingface/transformers/blob/b91e65afe0f467e24183928bf57d92b2cef4b69f/src/transformers/trainer.py#L256

class Trainer:
    from .trainer_pt_utils import _get_learning_rate, log_metrics, metrics_format, save_metrics, save_state

@A5rocks A5rocks mentioned this issue Oct 16, 2021
Closed
@matthen
Copy link

matthen commented Nov 25, 2021

Is there a way to configure mypy so that it will not follow imports into the transformers library alone?

@aphedges
Copy link

@matthen, yes, there is. Please see huggingface/transformers#13390 (comment).

@vikigenius vikigenius mentioned this issue Dec 12, 2021
Closed
hauntsaninja pushed a commit to hauntsaninja/mypy that referenced this issue Jan 20, 2022
Fix class scoped imports
8c61bb3 
Fixes python#11045, fixes huggingface/transformers#13390
Fixes python#10488
Fixes python#7045
Fixes python#7806
Fixes python#11641
Fixes python#11351
Fixes python#10488

Co-authored-by: @A5rocks
@hauntsaninja hauntsaninja mentioned this issue Jan 20, 2022
Merged
JelleZijlstra pushed a commit that referenced this issue Feb 17, 2022
@hauntsaninja
Fix crashes in class scoped imports (#12023)
1de5e55 
Fixes #11045, fixes huggingface/transformers#13390
Fixes #10488
Fixes #7045
Fixes #7806
Fixes #11641
Fixes #11351
Fixes #10488

Co-authored-by: @A5rocks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
crash
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix crashes in class scoped imports hauntsaninja/mypy
Expand imports in classes A5rocks/mypy
4 participants
@shirayu @matthen @davidhewitt @aphedges