Glob pattern in parser's option "project" slows down linting #2611

tosmolka · 2020-09-29T14:53:39Z

I have tried restarting my IDE and the issue persists.
I have updated to the latest version of the packages.
I have read the FAQ and my problem is not listed.

Repro

# Clone typescript-eslint into current directory
git clone https://github.com/typescript-eslint/typescript-eslint
cd typescript-eslint
# Install packages to increase number of files and subfolders in the directory
yarn install
# Lint ESLint rules with enabled debugging and pass tsconfig.json to project option via glob pattern
eslint packages\eslint-plugin\src\rules --parser-options debugLevel:true,project:**/packages/eslint-plugin/tsconfig.json

Expected Result

Linting with glob pattern should take similar time as when tsconfig.json is passed directly via relative path.

eslint typescript-eslint\packages\eslint-plugin\src\rules --parser-options debugLevel:true,project:packages/eslint-plugin/tsconfig.json

Output:

  typescript-eslint:typescript-estree:parser parserOptions.project (excluding ignored) matched projects: [ 'packages/eslint-plugin/tsconfig.json' ] +0ms
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: c:\src\github.com\typescript-eslint\packages\eslint-plugin\src\rules\adjacent-overload-signatures.ts +0ms
  typescript-eslint:typescript-estree:createWatchProgram File did not belong to any existing programs, moving to create/update. c:\src\github.com\typescript-eslint\packages\eslint-plugin\src\rules\adjacent-overload-signatures.ts +0ms
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for c:\src\github.com\typescript-eslint\packages\eslint-plugin\tsconfig.json. +7ms
  typescript-eslint:parser:parser Resolved libs from program: [ 'es2017' ] +0ms
  typescript-eslint:typescript-estree:parser parserOptions.project (excluding ignored) matched projects: [ 'packages/eslint-plugin/tsconfig.json' ] +5s
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: c:\src\github.com\typescript-eslint\packages\eslint-plugin\src\rules\array-type.ts +5s
  typescript-eslint:typescript-estree:createWatchProgram Found existing program for file. c:\src\github.com\typescript-eslint\packages\eslint-plugin\src\rules\array-type.ts +5s
  typescript-eslint:parser:parser Resolved libs from program: [ 'es2017' ] +328ms
  typescript-eslint:typescript-estree:parser parserOptions.project (excluding ignored) matched projects: [ 'packages/eslint-plugin/tsconfig.json' ] +722ms
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: c:\src\github.com\typescript-eslint\packages\eslint-plugin\src\rules\await-thenable.ts +722ms
  typescript-eslint:typescript-estree:createWatchProgram Found existing program for file. c:\src\github.com\typescript-eslint\packages\eslint-plugin\src\rules\await-thenable.ts +713ms
  typescript-eslint:parser:parser Resolved libs from program: [ 'es2017' ] +684ms
  typescript-eslint:typescript-estree:parser parserOptions.project (excluding ignored) matched projects: [ 'packages/eslint-plugin/tsconfig.json' ] +169ms
...
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: c:\src\github.com\typescript-eslint\packages\eslint-plugin\src\rules\typedef.ts +93ms
  typescript-eslint:typescript-estree:createWatchProgram Found existing program for file. c:\src\github.com\typescript-eslint\packages\eslint-plugin\src\rules\typedef.ts +95ms
  typescript-eslint:parser:parser Resolved libs from program: [ 'es2017' ] +89ms
  typescript-eslint:typescript-estree:parser parserOptions.project (excluding ignored) matched projects: [ 'packages/eslint-plugin/tsconfig.json' ] +141ms
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: c:\src\github.com\typescript-eslint\packages\eslint-plugin\src\rules\unbound-method.ts +142ms
  typescript-eslint:typescript-estree:createWatchProgram Found existing program for file. c:\src\github.com\typescript-eslint\packages\eslint-plugin\src\rules\unbound-method.ts +141ms
  typescript-eslint:parser:parser Resolved libs from program: [ 'es2017' ] +141ms
  typescript-eslint:typescript-estree:parser parserOptions.project (excluding ignored) matched projects: [ 'packages/eslint-plugin/tsconfig.json' ] +131ms
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: c:\src\github.com\typescript-eslint\packages\eslint-plugin\src\rules\unified-signatures.ts +130ms
  typescript-eslint:typescript-estree:createWatchProgram Found existing program for file. c:\src\github.com\typescript-eslint\packages\eslint-plugin\src\rules\unified-signatures.ts +130ms
  typescript-eslint:parser:parser Resolved libs from program: [ 'es2017' ] +142ms

Actual Result

Parser seems to be resolving glob pattern for every linted file individually which significantly increases time to finish the scan.

eslint packages\eslint-plugin\src\rules --parser-options debugLevel:true,project:**/packages/eslint-plugin/tsconfig.json

Output:

  typescript-eslint:typescript-estree:parser parserOptions.project (excluding ignored) matched projects: [ 'packages/eslint-plugin/tsconfig.json' ] +0ms
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: c:\src\github.com\typescript-eslint\packages\eslint-plugin\src\rules\adjacent-overload-signatures.ts +0ms
  typescript-eslint:typescript-estree:createWatchProgram File did not belong to any existing programs, moving to create/update. c:\src\github.com\typescript-eslint\packages\eslint-plugin\src\rules\adjacent-overload-signatures.ts +0ms
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for c:\src\github.com\typescript-eslint\packages\eslint-plugin\tsconfig.json. +0ms
  typescript-eslint:parser:parser Resolved libs from program: [ 'es2017' ] +0ms
  typescript-eslint:typescript-estree:parser parserOptions.project (excluding ignored) matched projects: [ 'packages/eslint-plugin/tsconfig.json' ] +11s
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: c:\src\github.com\typescript-eslint\packages\eslint-plugin\src\rules\array-type.ts +11s
  typescript-eslint:typescript-estree:createWatchProgram Found existing program for file. c:\src\github.com\typescript-eslint\packages\eslint-plugin\src\rules\array-type.ts +11s
  typescript-eslint:parser:parser Resolved libs from program: [ 'es2017' ] +5s
  typescript-eslint:typescript-estree:parser parserOptions.project (excluding ignored) matched projects: [ 'packages/eslint-plugin/tsconfig.json' ] +3s
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: c:\src\github.com\typescript-eslint\packages\eslint-plugin\src\rules\await-thenable.ts +3s
  typescript-eslint:typescript-estree:createWatchProgram Found existing program for file. c:\src\github.com\typescript-eslint\packages\eslint-plugin\src\rules\await-thenable.ts +3s
  typescript-eslint:parser:parser Resolved libs from program: [ 'es2017' ] +3s
  typescript-eslint:typescript-estree:parser parserOptions.project (excluding ignored) matched projects: [ 'packages/eslint-plugin/tsconfig.json' ] +3s
...
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: c:\src\github.com\typescript-eslint\packages\eslint-plugin\src\rules\typedef.ts +3s
  typescript-eslint:typescript-estree:createWatchProgram Found existing program for file. c:\src\github.com\typescript-eslint\packages\eslint-plugin\src\rules\typedef.ts +3s
  typescript-eslint:parser:parser Resolved libs from program: [ 'es2017' ] +3s
  typescript-eslint:typescript-estree:parser parserOptions.project (excluding ignored) matched projects: [ 'packages/eslint-plugin/tsconfig.json' ] +3s
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: c:\src\github.com\typescript-eslint\packages\eslint-plugin\src\rules\unbound-method.ts +3s
  typescript-eslint:typescript-estree:createWatchProgram Found existing program for file. c:\src\github.com\typescript-eslint\packages\eslint-plugin\src\rules\unbound-method.ts +3s
  typescript-eslint:parser:parser Resolved libs from program: [ 'es2017' ] +3s
  typescript-eslint:typescript-estree:parser parserOptions.project (excluding ignored) matched projects: [ 'packages/eslint-plugin/tsconfig.json' ] +4s
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: c:\src\github.com\typescript-eslint\packages\eslint-plugin\src\rules\unified-signatures.ts +4s
  typescript-eslint:typescript-estree:createWatchProgram Found existing program for file. c:\src\github.com\typescript-eslint\packages\eslint-plugin\src\rules\unified-signatures.ts +4s
  typescript-eslint:parser:parser Resolved libs from program: [ 'es2017' ] +4s

Additional Info

Until the issue is investigated and fixed we were suggesting adding a note in the documentation to warn users - #2602

Versions

package	version
`@typescript-eslint/typescript-estree`	`4.3.0`
`TypeScript`	`4.0.2`
`node`	`14.6.0`

The text was updated successfully, but these errors were encountered:

bradzacher · 2020-09-29T18:29:58Z

So looking at the glob you're using - it's always going to be slow.

**/packages/eslint-plugin/tsconfig.json is going to try to match (within this repo):

./packages/eslint-plugin/tsconfig.json
./.github/packages/eslint-plugin/tsconfig.json
./.vscode/packages/eslint-plugin/tsconfig.json
./docs/packages/eslint-plugin/tsconfig.json
./packages/eslint-plugin/tsconfig.json
./packages/eslint-plugin/packages/eslint-plugin/tsconfig.json
./packages/eslint-plugin/docs/packages/eslint-plugin/tsconfig.json
./packages/eslint-plugin/docs/rules/packages/eslint-plugin/tsconfig.json
./packages/eslint-plugin/src/packages/eslint-plugin/tsconfig.json
....

** is in general a terrible glob token to use because it means "any number of subfolders". In general using this will always cause you issues because JS projects (esp monorepos) usually have deep folder trees.

OTOH if you use a better glob - like the one we use in our project: ./packages/*/tsconfig.json then it has a smaller match and is lower impact.

Log dump with good glob

typescript-eslint:typescript-estree:parser parserOptions.project (excluding ignored) matched projects: [
  './tsconfig.eslint.json',
  './tests/integration/utils/jsconfig.json',
  './packages/eslint-plugin/tsconfig.json',
  './packages/eslint-plugin-internal/tsconfig.json',
  './packages/eslint-plugin-tslint/tsconfig.json',
  './packages/experimental-utils/tsconfig.json',
  './packages/parser/tsconfig.json',
  './packages/scope-manager/tsconfig.json',
  './packages/shared-fixtures/tsconfig.json',
  './packages/types/tsconfig.json',
  './packages/typescript-estree/tsconfig.json',
  './packages/visitor-keys/tsconfig.json'
] +12s
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: /Users/bradzacher/github/typescript-eslint/packages/eslint-plugin/src/configs/all.ts +12s
  typescript-eslint:typescript-estree:createWatchProgram Found existing program for file. /users/bradzacher/github/typescript-eslint/packages/eslint-plugin/src/configs/all.ts +879ms
  typescript-eslint:parser:parser Resolved libs from program: [ 'es2017' ] +96ms
  typescript-eslint:typescript-estree:parser parserOptions.project (excluding ignored) matched projects: [
  './tsconfig.eslint.json',
  './tests/integration/utils/jsconfig.json',
  './packages/eslint-plugin/tsconfig.json',
  './packages/eslint-plugin-internal/tsconfig.json',
  './packages/eslint-plugin-tslint/tsconfig.json',
  './packages/experimental-utils/tsconfig.json',
  './packages/parser/tsconfig.json',
  './packages/scope-manager/tsconfig.json',
  './packages/shared-fixtures/tsconfig.json',
  './packages/types/tsconfig.json',
  './packages/typescript-estree/tsconfig.json',
  './packages/visitor-keys/tsconfig.json'
] +31ms
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: /Users/bradzacher/github/typescript-eslint/packages/eslint-plugin/src/configs/base.ts +31ms
  typescript-eslint:typescript-estree:createWatchProgram Found existing program for file. /users/bradzacher/github/typescript-eslint/packages/eslint-plugin/src/configs/base.ts +29ms
  typescript-eslint:parser:parser Resolved libs from program: [ 'es2017' ] +20ms
  typescript-eslint:typescript-estree:parser parserOptions.project (excluding ignored) matched projects: [
  './tsconfig.eslint.json',
  './tests/integration/utils/jsconfig.json',
  './packages/eslint-plugin/tsconfig.json',
  './packages/eslint-plugin-internal/tsconfig.json',
  './packages/eslint-plugin-tslint/tsconfig.json',
  './packages/experimental-utils/tsconfig.json',
  './packages/parser/tsconfig.json',
  './packages/scope-manager/tsconfig.json',
  './packages/shared-fixtures/tsconfig.json',
  './packages/types/tsconfig.json',
.-
.-*.-
  './packages/typescript-estree/tsconfig.json',
  './packages/visitor-keys/tsconfig.json'
] +8ms
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: /Users/bradzacher/github/typescript-eslint/packages/eslint-plugin/src/configs/eslint-recommended.ts +8ms
  typescript-eslint:typescript-estree:createWatchProgram Found existing program for file. /users/bradzacher/github/typescript-eslint/packages/eslint-plugin/src/configs/eslint-recommended.ts +8ms
  typescript-eslint:parser:parser Resolved libs from program: [ 'es2017' ] +9ms
  typescript-eslint:typescript-estree:parser parserOptions.project (excluding ignored) matched projects: [
  './tsconfig.eslint.json',
  './tests/integration/utils/jsconfig.json',
  './packages/eslint-plugin/tsconfig.json',
  './packages/eslint-plugin-internal/tsconfig.json',
  './packages/eslint-plugin-tslint/tsconfig.json',
  './packages/experimental-utils/tsconfig.json',
  './packages/parser/tsconfig.json',
  './packages/scope-manager/tsconfig.json',
  './packages/shared-fixtures/tsconfig.json',
  './packages/types/tsconfig.json',
  './packages/typescript-estree/tsconfig.json',
  './packages/visitor-keys/tsconfig.json'
] +12ms

There will be a non-zero cost to globs OFC - globs require work to parse, expand and match against the file system.

Parser seems to be resolving glob pattern for every linted file individually

This is an unfortunate problem that we cannot work around.
ESLint does not tell the parser much about the run - it just says "here is a file to be parsed".
So we don't know if the file part of a larger lint run (eg npx eslint src/), or if it's part of a single-file lint run (eg npx eslint src/foo.ts), or if it's a single file as part of a continuous lint run (eg in an IDE).

The first two cases can be treated the same (the lint runs and then the instance dies), but the IDE case is special. The IDE is persistent, and reuses the parser again and again. This means the parser state is reused again and again.

Most parsers don't worry about this because they're stateless. But our parser is stateful because it's too expensive to recompute the TS type info. This means that our parser has to be built to handle the cases of:

when files are added.
when files are removed.
when eslint config changes.

When the ESLint config changes (i.e. you change parserOptions.project), we have to be able to respond to that appropriately.
When a file is added or removed, we have to update the TS programs appropriately (i.e. you add a new tsconfig.json which matches the glob you gave).

This is why we have to recompute the glob every single lint run.

tosmolka · 2020-09-30T07:41:24Z

Yes, this is why I suggested to update the docs as this seems like known limitation and is hard to fix on parser side.

bradzacher · 2020-09-30T08:15:41Z

An issue is important to understand the problem you're having and what the solutions are.
Without this discussion it was unclear what the issue was.
Often these things come purely down to user error - so I wanted to make sure that we weren't updating the docs because you'd just run into an issue yourself.

The problem here isn't that globs are slow - the problem is that unnecessarily wide globs are slow.
An appropriately crafted glob will be performant enough, as evidenced by our local use of a tight glob.

I'm okay if you add an FAQ article about appropriate globs and the slowness of **.
I don't think this belongs in the main parser docs. Those docs are long enough as it is - more text will only make them more confusing.

tosmolka · 2020-09-30T08:57:57Z

Absolutely, discussion about the issue is important. FAQ already has few notes around wide globs in "My linting feels really slow", we could maybe extend that. But we should also explicitly mention that glob needs to be recomputed for every parsed file. This is known limitation of parser but I don't think users generally expect that.

I was thinking about introducing a cache with reasonable expiration time but this seems overcomplicated and fragile.

How would you feel about collecting stats and maybe warning user in runtime when glob resolutions are slow (not yet sure about what metric to use) and suggesting to use less wide pattern(s) or path(s)?

Thanks.

bradzacher · 2020-09-30T18:00:15Z

I was thinking about introducing a cache with reasonable expiration time but this seems overcomplicated and fragile.

Yeah, I wish that the there was a way to do these queries at the filesystem level and have them cached by the filesystem.

Ultimately to make this work properly, we'll need to help ESLint evolve in this space so that it can tell us when a lint run starts and stops.
An RFC is on my todo list (eslint/eslint#13525)

How would you feel about collecting stats and maybe warning user in runtime when glob resolutions are slow (not yet sure about what metric to use) and suggesting to use less wide pattern(s) or path(s)?

I think this is fine to do - as long as we implement a way to turn off the log as well (similar to how we have warnOnUnsupportedTypeScriptVersion to turn off the message about being on an unsupported version.

tosmolka added package: typescript-estree Issues related to @typescript-eslint/typescript-estree triage Waiting for team members to take a look labels Sep 29, 2020

bradzacher added awaiting response Issues waiting for a reply from the OP or another party and removed triage Waiting for team members to take a look labels Sep 29, 2020

bradzacher added documentation Documentation ("docs") that needs adding/updating and removed awaiting response Issues waiting for a reply from the OP or another party labels Sep 30, 2020

JoshuaKGoldberg added the accepting prs Go ahead, send a pull request that resolves this issue label Oct 25, 2021

JoshuaKGoldberg added this to Documentation Nov 9, 2021

JoshuaKGoldberg moved this to Todo in Documentation Nov 9, 2021

JoshuaKGoldberg moved this from Todo to In Progress in Documentation Dec 30, 2021

JoshuaKGoldberg self-assigned this Dec 30, 2021

JoshuaKGoldberg mentioned this issue Dec 31, 2021

docs: mention deep globs in ESLint options in troubleshooting #4375

Merged

3 tasks

JoshuaKGoldberg moved this from In Progress to In Review in Documentation Dec 31, 2021

bradzacher closed this as completed in #4375 Dec 31, 2021

Repository owner moved this from In Review to Done in Documentation Dec 31, 2021

github-actions bot locked as resolved and limited conversation to collaborators Jan 31, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Glob pattern in parser's option "project" slows down linting #2611

Glob pattern in parser's option "project" slows down linting #2611

tosmolka commented Sep 29, 2020 •

edited

Loading

bradzacher commented Sep 29, 2020

tosmolka commented Sep 30, 2020

bradzacher commented Sep 30, 2020

tosmolka commented Sep 30, 2020 •

edited

Loading

bradzacher commented Sep 30, 2020

Glob pattern in parser's option "project" slows down linting #2611

Glob pattern in parser's option "project" slows down linting #2611

Comments

tosmolka commented Sep 29, 2020 • edited Loading

bradzacher commented Sep 29, 2020

tosmolka commented Sep 30, 2020

bradzacher commented Sep 30, 2020

tosmolka commented Sep 30, 2020 • edited Loading

bradzacher commented Sep 30, 2020

tosmolka commented Sep 29, 2020 •

edited

Loading

tosmolka commented Sep 30, 2020 •

edited

Loading