Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Incorporate jedi-language-server semantic highlights #25003

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
rivershah opened this issue Apr 22, 2025 · 6 comments · Fixed by #25006
Closed

Incorporate jedi-language-server semantic highlights #25003

rivershah opened this issue Apr 22, 2025 · 6 comments · Fixed by #25006
Labels
feature-request Request for new features or functionality triage-needed Needs assignment to the proper sub-team

Comments

@rivershah
Copy link

jedi-language-server now implements semantic highlights:

pappasam/jedi-language-server#336 (comment)

Can this please be incorporated. Previously discussed here:

#23049
@rivershah rivershah added the feature-request Request for new features or functionality label Apr 22, 2025
@github-actions github-actions bot added the triage-needed Needs assignment to the proper sub-team label Apr 22, 2025
@rivershah rivershah changed the title Incorporate jedi language server semantic highlights Incorporate jedi-language-server semantic highlights Apr 22, 2025
@karthiknadig
Copy link
Member

@rivershah Can you try the latest pre-release of the python extension? this should be included in it. I just added the latest Jedi yesterday.

@github-actions github-actions bot added the info-needed Issue requires more information from poster label Apr 22, 2025
@rivershah
Copy link
Author

@karthiknadig All works with pylance but not so with jedi. At the risk of too much info, here are logs with language server set to Pylance

2025-04-23 04:52:19.137 [info] (Client) Pylance async client (2025.4.1) started with python extension (2025.5.2025042201)
2025-04-23 04:52:19.746 [info] [Info  - 4:52:19 AM] (1465) Server root directory: file:///root/.vscode-server/extensions/ms-python.vscode-pylance-2025.4.1/dist
2025-04-23 04:52:19.748 [info] [Info  - 4:52:19 AM] (1465) Pylance language server 2025.4.1 (pyright version 1.1.398, commit 4f24eccc) starting
2025-04-23 04:52:19.770 [info] [Info  - 4:52:19 AM] (1465) Starting service instance "<project>" for workspace "/workspaces/<project>"
2025-04-23 04:52:19.877 [info] [Info  - 4:52:19 AM] (1465) Loading pyproject.toml file at /workspaces/<project>/pyproject.toml
2025-04-23 04:52:19.882 [info] [Info  - 4:52:19 AM] (1465) Setting environmentName for service "<project>": "3.12.10 (global)"
2025-04-23 04:52:19.883 [info] [Info  - 4:52:19 AM] (1465) Setting pythonPath for service "<project>": "/usr/local/bin/python"
2025-04-23 04:52:19.883 [info] [Info  - 4:52:19 AM] (1465) No include entries specified; assuming /workspaces/<project>
2025-04-23 04:52:19.884 [info] [Info  - 4:52:19 AM] (1465) Auto-excluding **/node_modules
2025-04-23 04:52:19.884 [info] [Info  - 4:52:19 AM] (1465) Auto-excluding **/__pycache__
2025-04-23 04:52:19.884 [info] [Info  - 4:52:19 AM] (1465) Auto-excluding **/.*
2025-04-23 04:52:19.950 [info] [Info  - 4:52:19 AM] (1465) Assuming Python version 3.12.10.final.0
2025-04-23 04:52:20.029 [info] [Info  - 4:52:19 AM] (1465) Found 17 source files
2025-04-23 04:52:20.079 [info] [Info  - 4:52:19 AM] (1465) Loading pyproject.toml file at /workspaces/<project>/pyproject.toml
2025-04-23 04:52:20.082 [info] [Info  - 4:52:19 AM] (1465) Setting environmentName for service "<project>": "3.12.10 (global)"
2025-04-23 04:52:20.083 [info] [Info  - 4:52:19 AM] (1465) Setting pythonPath for service "<project>": "/usr/local/bin/python"
2025-04-23 04:52:20.083 [info] [Info  - 4:52:19 AM] (1465) No include entries specified; assuming /workspaces/<project>
2025-04-23 04:52:20.084 [info] [Info  - 4:52:19 AM] (1465) Auto-excluding **/node_modules
2025-04-23 04:52:20.084 [info] [Info  - 4:52:19 AM] (1465) Auto-excluding **/__pycache__
2025-04-23 04:52:20.084 [info] [Info  - 4:52:19 AM] (1465) Auto-excluding **/.*
2025-04-23 04:52:20.084 [info] [Info  - 4:52:19 AM] (1465) Assuming Python version 3.12.10.final.0
2025-04-23 04:52:20.154 [info] [Info  - 4:52:19 AM] (1465) Found 17 source files
2025-04-23 04:52:20.567 [info] [Info  - 4:52:19 AM] (1465) BG: Priority queue background worker(2) root directory: file:///root/.vscode-server/extensions/ms-python.vscode-pylance-2025.4.1/dist
2025-04-23 04:52:20.569 [info] [Info  - 4:52:19 AM] (1465) BG: Priority queue background worker(2) started
2025-04-23 04:52:20.747 [info] [Info  - 4:52:19 AM] (1465) BG: Indexer background runner(3) root directory: file:///root/.vscode-server/extensions/ms-python.vscode-pylance-2025.4.1/dist (index)
2025-04-23 04:52:20.747 [info] [Info  - 4:52:19 AM] (1465) BG: Indexing(3) started
2025-04-23 04:52:21.456 [info] [Info  - 4:52:19 AM] (1465) BG: scanned(3) 149 files over 1 exec env
2025-04-23 04:52:23.728 [info] [Info  - 4:52:19 AM] (1465) BG: [IDX(3)] Long operation: index execution environment file:///workspaces/<project> (2031ms)
2025-04-23 04:52:23.756 [info] [Info  - 4:52:19 AM] (1465) BG: [IDX(3)] Long operation: index packages file:///workspaces/<project> (2070ms)
2025-04-23 04:52:23.756 [info] [Info  - 4:52:19 AM] (1465) BG: indexed(3) 149 files over 1 exec env
2025-04-23 04:52:23.816 [info] [Info  - 4:52:19 AM] (1465) BG: Indexing finished(3).
2025-04-23 04:52:42.038 [info] [Info  - 4:52:19 AM] (1465) BG: Indexer background runner(4) root directory: file:///root/.vscode-server/extensions/ms-python.vscode-pylance-2025.4.1/dist (refresh)
2025-04-23 04:52:42.038 [info] [Info  - 4:52:19 AM] (1465) BG: Indexing(4) started
2025-04-23 04:52:42.465 [info] [Info  - 4:52:19 AM] (1465) BG: scanned(4) 149 files over 1 exec env
2025-04-23 04:52:42.713 [info] [Info  - 4:52:19 AM] (1465) BG: indexed(4) 149 files over 1 exec env
2025-04-23 04:52:42.767 [info] [Info  - 4:52:19 AM] (1465) BG: Indexing finished(4).

Here are logs with language server set to Jedi (semantic highlights don't work).

2025-04-23 04:54:48.661 [info] semantic_tokens_full <file>
2025-04-23 04:54:48.666 [info] semantic_tokens_range <file> 0:0-69:0
2025-04-23 04:54:50.519 [info] [Debug - 4:54:50 AM] multiple definitions found for name "path" of type 'statement' (7:19):
 <Name full_name='os.path', description='module ntpath'>
 <Name full_name='os.path', description='module posixpath'>
2025-04-23 04:54:50.521 [info] [Debug - 4:54:50 AM] multiple definitions found for name "path" of type 'statement' (7:19):
 <Name full_name='os.path', description='module posixpath'>
 <Name full_name='os.path', description='module ntpath'>
…  
2025-04-23 04:54:51.431 [info] ERROR:pygls.protocol.json_rpc:Exception occurred for message "4": ValueError: not enough values to unpack (expected 1, got 0)
2025-04-23 04:54:51.438 [info] [Error - 4:54:51 AM] Request textDocument/semanticTokens/full failed.
2025-04-23 04:54:51.438 [info]   Message: ValueError: not enough values to unpack (expected 1, got 0)
2025-04-23 04:54:51.757 [info] ERROR:pygls.protocol.json_rpc:Exception occurred for message "5": AssertionError
2025-04-23 04:54:51.758 [info] [Error - 4:54:51 AM] Request textDocument/semanticTokens/range failed.
2025-04-23 04:54:51.758 [info]   Message: AssertionError
…  
2025-04-23 04:56:00.439 [info] semantic_tokens_range <file> 0:0-69:0
2025-04-23 04:56:00.454 [info] ERROR:pygls.protocol.json_rpc:Exception occurred for message "7": ValueError: not enough values to unpack (expected 1, got 0)
…  
2025-04-23 04:56:04.629 [info] ERROR:pygls.protocol.json_rpc:Failed to handle request 11 textDocument/documentHighlight DocumentHighlightParams(text_document=TextDocumentIdentifier(uri='<file>'), position=10:60)
…  
2025-04-23 04:56:04.633 [info] [Error - 4:56:04 AM] Request textDocument/documentHighlight failed.
2025-04-23 04:56:04.633 [info]   Message: ValueError: not enough values to unpack (expected 1, got 0)
…  
2025-04-23 04:56:06.129 [info] semantic_tokens_full <file>
2025-04-23 04:56:06.297 [info] [Error - 4:56:06 AM] Request textDocument/semanticTokens/full failed.
2025-04-23 04:56:06.331 [info] ERROR:pygls.protocol.json_rpc:Failed to handle request 21 textDocument/documentHighlight DocumentHighlightParams(text_document=TextDocumentIdentifier(uri='<file>'), position=75:4)
…  
2025-04-23 04:56:06.332 [info] [Error - 4:56:06 AM] Request textDocument/documentHighlight failed.
2025-04-23 04:56:06.333 [info]   Message: AssertionError
…  
2025-04-23 04:56:06.366 [info] semantic_tokens_range <file> 24:0-116:0
2025-04-23 04:56:06.381 [info] [Error - 4:56:06 AM] Request textDocument/semanticTokens/range failed.
…  
2025-04-23 04:56:07.411 [info] semantic_tokens_full <file>
2025-04-23 04:56:07.421 [info] semantic_tokens_range <file> 0:0-69:0
2025-04-23 04:56:07.426 [info] [Error - 4:56:07 AM] Request textDocument/semanticTokens/full failed.

@github-actions github-actions bot removed the info-needed Issue requires more information from poster label Apr 23, 2025
@karthiknadig karthiknadig mentioned this issue Apr 23, 2025
Merged
@karthiknadig
Copy link
Member

The fix was that this was not enabled when we sent the options to JediLS:

Image

@github-actions github-actions bot added the info-needed Issue requires more information from poster label Apr 23, 2025
@karthiknadig
Copy link
Member

Verification steps:

  1. set python.languageServer setting to Jedi
  2. Looks at Output > Python Langauge Server logs for semantic tokens

@rivershah
Copy link
Author

Now I see this please. Server also crashes after this error:

2025-04-24 12:13:53.988 [info] [Error - 12:13:53 PM] Request textDocument/semanticTokens/range failed.
2025-04-24 12:13:53.988 [info]   Message: _pickle.UnpicklingError: invalid load key, ' '.
  Code: -32603 
[object Object]
2025-04-24 12:13:54.844 [info] ERROR:pygls.protocol.json_rpc:Failed to handle request 31 textDocument/codeAction CodeActionParams(text_document=TextDocumentIdentifier(uri='file:///[REDACTED_PATH]'), range=100:35-100:35, context=CodeActionContext(diagnostics=[], only=None, trigger_kind=<CodeActionTriggerKind.Automatic: 2>), work_done_token=None, partial_result_token=None)
Traceback (most recent call last):
  File "[REDACTED]/json_rpc.py", line 266, in _handle_request
    self._execute_request(msg_id, handler, params)
  File "[REDACTED]/json_rpc.py", line 188, in _execute_request
    self._send_response(msg_id, handler(params))
                                ^^^^^^^^^^^^^^^
  File "[REDACTED]/notebook_utils.py", line 373, in wrapped
    result = f(notebook_server, notebook_params)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/server.py", line 717, in code_action
    extract_function_refactoring = jedi_script.extract_function(
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/helpers.py", line 487, in wrapper
    return func(self, line, column, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/__init__.py", line 675, in extract_function
    return extract_function(
           ^^^^^^^^^^^^^^^^^
  File "[REDACTED]/extract.py", line 216, in extract_function
    params, return_variables = list(_find_inputs_and_outputs(module_context, context, nodes))
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/extract.py", line 332, in _find_inputs_and_outputs
    name_definitions = context.goto(name, name.start_pos)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/context.py", line 34, in goto
    names = finder.filter_name(filters, name_or_str)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/finder.py", line 36, in filter_name
    names = filter.get(string_name)
            ^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/filters.py", line 109, in get
    return self._convert_names(self._filter(
                               ^^^^^^^^^^^^^
  File "[REDACTED]/filters.py", line 145, in _filter
    return list(self._check_flows(names))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/filters.py", line 156, in _check_flows
    check = flow_analysis.reachability_check(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/flow_analysis.py", line 85, in reachability_check
    return _break_check(context, value_scope, first_flow_scope, node)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/flow_analysis.py", line 93, in _break_check
    reachable = _check_if(context, check_node)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/flow_analysis.py", line 120, in _check_if
    types = context.infer_node(node)
            ^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/context.py", line 224, in infer_node
    return infer_node(self, node)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/syntax_tree.py", line 157, in infer_node
    return _infer_node_if_inferred(context, element)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/syntax_tree.py", line 170, in _infer_node_if_inferred
    return _infer_node_cached(context, element)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/cache.py", line 44, in wrapper
    rv = function(obj, *args, **kwargs)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/syntax_tree.py", line 175, in _infer_node_cached
    return _infer_node(context, element)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/debug.py", line 81, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/syntax_tree.py", line 83, in wrapper
    return func(context, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/syntax_tree.py", line 218, in _infer_node
    value_set = context.infer_node(element.children[-1])
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/context.py", line 224, in infer_node
    return infer_node(self, node)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/syntax_tree.py", line 157, in infer_node
    return _infer_node_if_inferred(context, element)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/syntax_tree.py", line 170, in _infer_node_if_inferred
    return _infer_node_cached(context, element)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/cache.py", line 44, in wrapper
    rv = function(obj, *args, **kwargs)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/syntax_tree.py", line 175, in _infer_node_cached
    return _infer_node(context, element)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/debug.py", line 81, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/syntax_tree.py", line 185, in _infer_node
    return infer_atom(context, element)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/syntax_tree.py", line 305, in infer_atom
    return context.py__getattribute__(atom, position=position)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/context.py", line 77, in py__getattribute__
    values = ValueSet.from_sets(name.infer() for name in names)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/base_value.py", line 430, in from_sets
    for set_ in sets:
                ^^^^
  File "[REDACTED]/context.py", line 77, in <genexpr>
    values = ValueSet.from_sets(name.infer() for name in names)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/names.py", line 281, in infer
    return tree_name_to_values(
           ^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/__init__.py", line 21, in wrapper
    return built_functions[public_name](*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/stdlib.py", line 878, in wrapper
    return func(inference_state, context, tree_name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/django.py", line 177, in wrapper
    result = func(inference_state, context, tree_name)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/syntax_tree.py", line 771, in tree_name_to_values
    types = imports.infer_import(context, tree_name)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/cache.py", line 44, in wrapper
    rv = function(obj, *args, **kwargs)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/imports.py", line 53, in infer_import
    _prepare_infer_import(module_context, tree_name)
  File "[REDACTED]/imports.py", line 116, in _prepare_infer_import
    return from_import_name, tuple(import_path), import_node.level, importer.follow()
                                                                    ^^^^^^^^^^^^^^^^^
  File "[REDACTED]/imports.py", line 296, in follow
    return import_module_by_names(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/imports.py", line 377, in import_module_by_names
    import_module(
  File "[REDACTED]/__init__.py", line 21, in wrapper
    return built_functions[public_name](*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/flask.py", line 20, in wrapper
    return callback(inference_state, import_names, module_context, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/typeshed.py", line 115, in wrapper
    python_value_set = ValueSet.from_sets(
                       ^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/imports.py", line 411, in import_module
    file_io_or_ns, is_pkg = inference_state.compiled_subprocess.get_module_info(
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/subprocess.py", line 188, in wrapper
    result = self._compiled_subprocess.run(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/subprocess.py", line 293, in run
    self._send(delete_id, None)
  File "[REDACTED]/subprocess.py", line 318, in _send
    is_exception, traceback, result = pickle_load(self._get_process().stdout)
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/_compatibility.py", line 25, in pickle_load
    return Unpickler(file).load()
           ^^^^^^^^^^^^^^^^^^^^^^
_pickle.UnpicklingError: invalid load key, ' '.

2025-04-24 12:13:54.847 [info] [Error - 12:13:54 PM] Request textDocument/codeAction failed.
2025-04-24 12:13:54.847 [info]   Message: _pickle.UnpicklingError: invalid load key, ' '.
  Code: -32603 
[object Object]

@github-actions github-actions bot removed the info-needed Issue requires more information from poster label Apr 24, 2025
@karthiknadig
Copy link
Member

karthiknadig commented Apr 24, 2025 •
edited
Loading

This should be filed on jedi-language-server. All we do is launch it with settings.

We can look into adding settings for Jedi so this can be controlled this easily.

@rivershah rivershah mentioned this issue Apr 25, 2025
Closed
@pappasam pappasam mentioned this issue Apr 25, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
feature-request Request for new features or functionality triage-needed Needs assignment to the proper sub-team
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: enable semantic tokens in Jedi language server analysis options karthiknadig/vscode-python
2 participants
@karthiknadig @rivershah