parser,checker: use keywords matcher trie for imported symbol presence checks, instead of `x in p.imported_symbol` #25201

spytheman · 2025-08-30T16:49:42Z

The p.imported_symbol map is usually very short, and does not change after the import section of a file is processed.

parser: use a token.KeywordsMatcherTrie matching checks, instead of the map checks for name in p.imported_symbols
checker: use an imported symbols trie stored per each ast.File too
v.token: add manual .str() methods for KeywordsMatcherTrie and TrieNode, to workaround cgen errors
parser: fix forgotten x in p.imported_symbols expression in fn.v

…he map checks for `name in p.imported_symbols`

…de, to workaround cgen errors

huly-for-github · 2025-08-30T16:50:05Z

Connected to Huly®: V_0.6-24386

spytheman · 2025-08-30T16:53:59Z

The parser here is a bit faster compared to the one on master:

#0 19:22:34 ^ parser_use_keywords_matcher_trie_for_imported_symbol_checks ~/v>v run .github/workflows/compare_pr_to_master.v
==================================================================================================
Current git branch: parser_use_keywords_matcher_trie_for_imported_symbol_checks, commit: 834dba8
    Compiling new V executables from PR branch: parser_use_keywords_matcher_trie_for_imported_symbol_checks, commit: 834dba8 ...
CPU: 2.79s      Real: 3.01s     Elapsed: 0:03.01        RAM: 425072KB   ./v -o vnew1 cmd/v
CPU: 2.77s      Real: 3.01s     Elapsed: 0:03.01        RAM: 425064KB   ./vnew1 -o vnew2 cmd/v
CPU: 2.79s      Real: 3.01s     Elapsed: 0:03.01        RAM: 425056KB   ./vnew2 -no-parallel -o vnew cmd/v
CPU: 0.17s      Real: 0.20s     Elapsed: 0:00.20        RAM: 46760KB    ./vnew -no-parallel -o nhw_current.c examples/hello_world.v
CPU: 0.18s      Real: 0.20s     Elapsed: 0:00.20        RAM: 47112KB    ./vnew -no-parallel -o nhw_current_gcc.c -cc gcc examples/hello_world.v
CPU: 2.39s      Real: 2.58s     Elapsed: 0:02.58        RAM: 425052KB   ./vnew -no-parallel -o nv_current.c cmd/v
>Size of        nhw_current.c:      28421
>Size of    nhw_current_gcc.c:      42044
>Size of         nv_current.c:    6423953
>Size of                 vnew:   10045890
Switched to branch 'v_repo_master'
Your branch is up to date with 'V_REPO/master'.
==================================================================================================
    Compiling old V executables from branch: v_repo_master, commit: 2ac3478 ...
CPU: 2.79s      Real: 3.00s     Elapsed: 0:03.00        RAM: 424824KB   ./v -o vold1 cmd/v
CPU: 2.77s      Real: 3.02s     Elapsed: 0:03.02        RAM: 423992KB   ./vold1 -o vold2 cmd/v
CPU: 2.79s      Real: 3.02s     Elapsed: 0:03.02        RAM: 424000KB   ./vold2 -no-parallel -o vold cmd/v
CPU: 0.18s      Real: 0.20s     Elapsed: 0:00.20        RAM: 46600KB    ./vold -no-parallel -o ohw_master.c examples/hello_world.v
CPU: 0.19s      Real: 0.20s     Elapsed: 0:00.20        RAM: 46952KB    ./vold -no-parallel -o ohw_master_gcc.c -cc gcc examples/hello_world.v
CPU: 2.38s      Real: 2.59s     Elapsed: 0:02.59        RAM: 424012KB   ./vold -no-parallel -o ov_master.c cmd/v
>Size of         ohw_master.c:      28421
>Size of     ohw_master_gcc.c:      42044
>Size of          ov_master.c:    6419744
>Size of                 vold:   10044613
==================================================================================================
File sizes so far ...
>>>>>> size("    nhw_current.c") - size("     ohw_master.c") =      28421 -      28421 =          0
>>>>>> size("nhw_current_gcc.c") - size(" ohw_master_gcc.c") =      42044 -      42044 =          0
>>>>>> size("     nv_current.c") - size("      ov_master.c") =    6423953 -    6419744 =       4209
>>>>>> size("             vnew") - size("             vold") =   10045890 -   10044613 =       1277
Switched to branch 'parser_use_keywords_matcher_trie_for_imported_symbol_checks'
==================================================================================================
    Measuring at PR branch: parser_use_keywords_matcher_trie_for_imported_symbol_checks, commit: 834dba8 ...
Summary after 1 series x 10 runs (%s are relative to first command, or `base`), discard maxs:  7, repeat: 1/3, took: 2912.211 ms                                                              
    1     -1.0%   1.01x faster  119.8ms ± σ:    0.4ms,  119.4ms… 120.5ms `./vnew -check-syntax           examples/hello_world.v`
 >  2           base            121.0ms ± σ:    0.2ms,  120.8ms… 121.3ms `./vold -check-syntax           examples/hello_world.v`
Summary after 1 series x 10 runs (%s are relative to first command, or `base`), discard maxs:  7, repeat: 2/3, took: 2910.645 ms                                                              
    1     -1.2%   1.01x faster  119.7ms ± σ:    0.2ms,  119.5ms… 119.9ms `./vnew -check-syntax           examples/hello_world.v`
 >  2           base            121.1ms ± σ:    0.2ms,  120.8ms… 121.3ms `./vold -check-syntax           examples/hello_world.v`
Summary after 1 series x 10 runs (%s are relative to first command, or `base`), discard maxs:  7, repeat: 3/3, took: 2914.737 ms                                                              
    1     -1.0%   1.01x faster  120.0ms ± σ:    0.1ms,  119.9ms… 120.0ms `./vnew -check-syntax           examples/hello_world.v`
 >  2           base            121.2ms ± σ:    0.3ms,  120.9ms… 121.6ms `./vold -check-syntax           examples/hello_world.v`
Summary after 1 series x 10 runs (%s are relative to first command, or `base`), discard maxs:  7, repeat: 1/3, took: 4420.643 ms                                                              
    1     -0.8%   1.01x faster  182.4ms ± σ:    0.3ms,  182.0ms… 182.7ms `./vnew -check                  examples/hello_world.v`
 >  2           base            183.8ms ± σ:    0.4ms,  183.4ms… 184.3ms `./vold -check                  examples/hello_world.v`
Summary after 1 series x 10 runs (%s are relative to first command, or `base`), discard maxs:  7, repeat: 2/3, took: 4471.713 ms                                                              
    1     -1.3%   1.01x faster  181.9ms ± σ:    0.2ms,  181.6ms… 182.1ms `./vnew -check                  examples/hello_world.v`
 >  2           base            184.3ms ± σ:    0.1ms,  184.2ms… 184.4ms `./vold -check                  examples/hello_world.v`
Summary after 1 series x 10 runs (%s are relative to first command, or `base`), discard maxs:  7, repeat: 3/3, took: 4421.540 ms                                                              
    1     -0.9%   1.01x faster  181.5ms ± σ:    0.5ms,  180.8ms… 182.2ms `./vnew -check                  examples/hello_world.v`
 >  2           base            183.1ms ± σ:    0.4ms,  182.6ms… 183.6ms `./vold -check                  examples/hello_world.v`
Summary after 1 series x 10 runs (%s are relative to first command, or `base`), discard maxs:  7, repeat: 1/3, took: 4902.346 ms                                                              
    1     -1.0%   1.01x faster  201.2ms ± σ:    0.3ms,  200.8ms… 201.6ms `./vnew -no-parallel -o nhw.c   examples/hello_world.v`
 >  2           base            203.3ms ± σ:    0.1ms,  203.2ms… 203.4ms `./vold -no-parallel -o ohw.c   examples/hello_world.v`
Summary after 1 series x 10 runs (%s are relative to first command, or `base`), discard maxs:  7, repeat: 2/3, took: 4877.540 ms                                                              
    1     -0.9%   1.01x faster  201.4ms ± σ:    0.4ms,  201.0ms… 201.9ms `./vnew -no-parallel -o nhw.c   examples/hello_world.v`
 >  2           base            203.3ms ± σ:    0.4ms,  202.8ms… 203.8ms `./vold -no-parallel -o ohw.c   examples/hello_world.v`
Summary after 1 series x 10 runs (%s are relative to first command, or `base`), discard maxs:  7, repeat: 3/3, took: 4870.346 ms                                                              
    1     -1.0%   1.01x faster  200.8ms ± σ:    0.2ms,  200.5ms… 200.9ms `./vnew -no-parallel -o nhw.c   examples/hello_world.v`
 >  2           base            202.7ms ± σ:    0.2ms,  202.5ms… 203.0ms `./vold -no-parallel -o ohw.c   examples/hello_world.v`
>>>>>> size("            nhw.c") - size("            ohw.c") =      28421 -      28421 =          0
Summary after 1 series x 10 runs (%s are relative to first command, or `base`), discard maxs:  7, repeat: 1/3, took: 5139.237 ms                                                              
    1     -0.6%   1.01x faster  212.2ms ± σ:    0.2ms,  211.9ms… 212.4ms `./vnew -no-parallel -o nhw.exe examples/hello_world.v`
 >  2           base            213.5ms ± σ:    0.5ms,  212.8ms… 213.9ms `./vold -no-parallel -o ohw.exe examples/hello_world.v`
Summary after 1 series x 10 runs (%s are relative to first command, or `base`), discard maxs:  7, repeat: 2/3, took: 5149.537 ms                                                              
    1     -0.5%   1.00x faster  212.3ms ± σ:    0.0ms,  212.2ms… 212.4ms `./vnew -no-parallel -o nhw.exe examples/hello_world.v`
 >  2           base            213.3ms ± σ:    0.3ms,  213.0ms… 213.6ms `./vold -no-parallel -o ohw.exe examples/hello_world.v`
Summary after 1 series x 10 runs (%s are relative to first command, or `base`), discard maxs:  7, repeat: 3/3, took: 5146.021 ms                                                              
    1     -0.5%   1.01x faster  212.4ms ± σ:    0.6ms,  211.5ms… 213.0ms `./vnew -no-parallel -o nhw.exe examples/hello_world.v`
 >  2           base            213.5ms ± σ:    0.6ms,  212.6ms… 214.0ms `./vold -no-parallel -o ohw.exe examples/hello_world.v`
>>>>>> size("          nhw.exe") - size("          ohw.exe") =     241245 -     241245 =          0
Summary after 1 series x 10 runs (%s are relative to first command, or `base`), discard maxs:  7, repeat: 1/3, took: 23718.173 ms                                                             
    1     -1.1%   1.01x faster  975.7ms ± σ:    0.9ms,  974.4ms… 976.4ms `./vnew -check-syntax           cmd/v`
 >  2           base            986.9ms ± σ:    1.2ms,  985.2ms… 988.1ms `./vold -check-syntax           cmd/v`
Summary after 1 series x 10 runs (%s are relative to first command, or `base`), discard maxs:  7, repeat: 2/3, took: 23628.506 ms                                                             
    1     -1.1%   1.01x faster  975.8ms ± σ:    1.3ms,  973.9ms… 977.0ms `./vnew -check-syntax           cmd/v`
 >  2           base            986.2ms ± σ:    1.0ms,  985.2ms… 987.6ms `./vold -check-syntax           cmd/v`
Summary after 1 series x 10 runs (%s are relative to first command, or `base`), discard maxs:  7, repeat: 3/3, took: 23726.371 ms                                                             
    1     -1.1%   1.01x faster  977.3ms ± σ:    0.2ms,  977.1ms… 977.6ms `./vnew -check-syntax           cmd/v`
 >  2           base            988.0ms ± σ:    1.5ms,  986.1ms… 989.8ms `./vold -check-syntax           cmd/v`
Summary after 1 series x 10 runs (%s are relative to first command, or `base`), discard maxs:  7, repeat: 1/3, took: 40260.327 ms                                                             
    1     -0.7%   1.01x faster 1664.5ms ± σ:    1.9ms, 1662.3ms…1667.0ms `./vnew -check                  cmd/v`
 >  2           base           1677.0ms ± σ:    0.1ms, 1676.8ms…1677.1ms `./vold -check                  cmd/v`
Summary after 1 series x 10 runs (%s are relative to first command, or `base`), discard maxs:  7, repeat: 2/3, took: 40399.261 ms                                                             
    1     -0.7%   1.01x faster 1666.2ms ± σ:    2.5ms, 1662.8ms…1668.6ms `./vnew -check                  cmd/v`
 >  2           base           1677.6ms ± σ:    1.7ms, 1675.2ms…1679.0ms `./vold -check                  cmd/v`
Summary after 1 series x 10 runs (%s are relative to first command, or `base`), discard maxs:  7, repeat: 3/3, took: 40288.511 ms                                                             
    1     -0.5%   1.00x faster 1667.1ms ± σ:    0.6ms, 1666.3ms…1667.5ms `./vnew -check                  cmd/v`
 >  2           base           1674.9ms ± σ:    1.3ms, 1673.1ms…1676.1ms `./vold -check                  cmd/v`
Summary after 1 series x 10 runs (%s are relative to first command, or `base`), discard maxs:  7, repeat: 1/3, took: 62298.040 ms                                                             
    1     -0.2%   1.00x faster 2586.2ms ± σ:    0.9ms, 2585.0ms…2587.2ms `./vnew -no-parallel -o nv.c    cmd/v`
 >  2           base           2592.6ms ± σ:    2.2ms, 2590.1ms…2595.6ms `./vold -no-parallel -o ov.c    cmd/v`
Summary after 1 series x 10 runs (%s are relative to first command, or `base`), discard maxs:  7, repeat: 2/3, took: 62287.900 ms
    1     -0.3%   1.00x faster 2583.1ms ± σ:    0.8ms, 2582.0ms…2583.8ms `./vnew -no-parallel -o nv.c    cmd/v`
 >  2           base           2590.5ms ± σ:    0.4ms, 2590.1ms…2591.0ms `./vold -no-parallel -o ov.c    cmd/v`
Summary after 1 series x 10 runs (%s are relative to first command, or `base`), discard maxs:  7, repeat: 3/3, took: 62347.913 ms
    1     -0.3%   1.00x faster 2580.7ms ± σ:    1.7ms, 2578.5ms…2582.8ms `./vnew -no-parallel -o nv.c    cmd/v`
 >  2           base           2588.9ms ± σ:    3.1ms, 2585.2ms…2592.8ms `./vold -no-parallel -o ov.c    cmd/v`
>>>>>> size("             nv.c") - size("             ov.c") =    6423953 -    6423953 =          0
Summary after 1 series x 10 runs (%s are relative to first command, or `base`), discard maxs:  7, repeat: 1/3, took: 72469.225 ms
    1     -0.3%   1.00x faster 3008.6ms ± σ:    4.5ms, 3002.3ms…3012.6ms `./vnew -no-parallel -o nv.exe  cmd/v`
 >  2           base           3017.4ms ± σ:    1.8ms, 3015.1ms…3019.6ms `./vold -no-parallel -o ov.exe  cmd/v`
Summary after 1 series x 10 runs (%s are relative to first command, or `base`), discard maxs:  7, repeat: 2/3, took: 72420.860 ms
    1     -0.2%   1.00x faster 3003.3ms ± σ:    2.7ms, 2999.5ms…3005.4ms `./vnew -no-parallel -o nv.exe  cmd/v`
 >  2           base           3009.0ms ± σ:    3.4ms, 3004.2ms…3011.5ms `./vold -no-parallel -o ov.exe  cmd/v`
Summary after 1 series x 10 runs (%s are relative to first command, or `base`), discard maxs:  7, repeat: 3/3, took: 71837.693 ms
    1     -0.3%   1.00x faster 2978.6ms ± σ:    4.0ms, 2975.5ms…2984.3ms `./vnew -no-parallel -o nv.exe  cmd/v`
 >  2           base           2986.3ms ± σ:    2.9ms, 2982.2ms…2988.6ms `./vold -no-parallel -o ov.exe  cmd/v`
>>>>>> size("           nv.exe") - size("           ov.exe") =   10045892 -   10045892 =          0
Done. Total time: 673.234120596 s.
==================================================================================================
Final summary for file diff sizes on their own branches:
>>>>>> size("    nhw_current.c") - size("     ohw_master.c") =      28421 -      28421 =          0
>>>>>> size("nhw_current_gcc.c") - size(" ohw_master_gcc.c") =      42044 -      42044 =          0
>>>>>> size("     nv_current.c") - size("      ov_master.c") =    6423953 -    6419744 =       4209
>>>>>> size("             vnew") - size("             vold") =   10045890 -   10044613 =       1277
==================================================================================================
Final summary for file diff sizes for generated files on the *current* branch:
>>>>>> size("            nhw.c") - size("            ohw.c") =      28421 -      28421 =          0
>>>>>> size("             nv.c") - size("             ov.c") =    6423953 -    6423953 =          0
>>>>>> size("          nhw.exe") - size("          ohw.exe") =     241245 -     241245 =          0
>>>>>> size("           nv.exe") - size("           ov.exe") =   10045892 -   10045892 =          0
#0 19:33:52 ^ parser_use_keywords_matcher_trie_for_imported_symbol_checks ~/v>git status
On branch parser_use_keywords_matcher_trie_for_imported_symbol_checks
nothing to commit, working tree clean
#0 19:49:33 ^ parser_use_keywords_matcher_trie_for_imported_symbol_checks ~/v>gh pr create --fill

spytheman added 4 commits August 30, 2025 18:58

parser: use a token.KeywordsMatcherTrie matching checks, instead of t…

b588bac

…he map checks for `name in p.imported_symbols`

checker: use an imported symbols trie stored per each ast.File too

1b3ec4a

v.token: add manual .str() methods for KeywordsMatcherTrie and TrieNo…

4bc7283

…de, to workaround cgen errors

parser: fix forgotten x in p.imported_symbols expression in fn.v

834dba8

spytheman changed the title ~~parser use keywords matcher trie for imported symbol checks~~ parser,checker: use keywords matcher trie for imported symbol presence checks, instead of x in p.imported_symbol (that map is usually very short, and does not change after the import section) Aug 30, 2025

run v fmt -w vlib/v/ast/ast.v

7abe91d

spytheman merged commit cb2756e into vlang:master Aug 30, 2025
83 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

parser,checker: use keywords matcher trie for imported symbol presence checks, instead of `x in p.imported_symbol` #25201

parser,checker: use keywords matcher trie for imported symbol presence checks, instead of `x in p.imported_symbol` #25201

Uh oh!

spytheman commented Aug 30, 2025 •

edited

Loading

Uh oh!

huly-for-github bot commented Aug 30, 2025

Uh oh!

spytheman commented Aug 30, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

parser,checker: use keywords matcher trie for imported symbol presence checks, instead of x in p.imported_symbol #25201

parser,checker: use keywords matcher trie for imported symbol presence checks, instead of x in p.imported_symbol #25201

Uh oh!

Conversation

spytheman commented Aug 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

huly-for-github bot commented Aug 30, 2025

Uh oh!

spytheman commented Aug 30, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

parser,checker: use keywords matcher trie for imported symbol presence checks, instead of `x in p.imported_symbol` #25201

parser,checker: use keywords matcher trie for imported symbol presence checks, instead of `x in p.imported_symbol` #25201

spytheman commented Aug 30, 2025 •

edited

Loading