From 950dec7452ee9abbccbfbf75d5ef1786dad47cab Mon Sep 17 00:00:00 2001 From: Brad Zacher Date: Tue, 10 Jan 2023 18:12:55 +1030 Subject: [PATCH 1/3] docs: document import/extensions slowness --- docs/linting/Troubleshooting.md | 63 +++++++++++++++++++++++++++++++++ 1 file changed, 63 insertions(+) diff --git a/docs/linting/Troubleshooting.md b/docs/linting/Troubleshooting.md index f1aff9346bb5..91fb816021bb 100644 --- a/docs/linting/Troubleshooting.md +++ b/docs/linting/Troubleshooting.md @@ -311,6 +311,69 @@ The following rules do not have equivalent checks in TypeScript, so we recommend - `import/no-unused-modules` - `import/no-deprecated` +#### `import/extensions` + +There are two uses for this rule: + +1. to enforce file extensions are always used, +2. to enforce file extensions are never used. + +##### Enforcing extensions are used + +If you want to enforce file extensions are always used and you're **NOT** using `moduleResolution` `node16` or `nodenext`, then there's not really a good alternative for you, and you should continue using the `import/extensions` lint rule. + +If you want to enforce file extensions are always used and you **ARE** using `moduleResolution` `node16` or `nodenext`, then you don't need to use the lint rule at all because TypeScript will automatically enforce that you include extensions (TS 2834 & 2835)! + +##### Enforcing extensions are not used + +On the surface `import/extensions` seems like it should be fast for this use case, however the rule isn't just a pure AST-check - it has to resolve modules on disk so that it doesn't false positive on cases where you are importing modules with an extension as part of their name (eg `foo.js` resolves to `node_modules/foo.js/index.js`, so the `.js` is required). This disk lookup is costly and thus makes the rule slow. + +If your project doesn't use any `npm` packages with a file extension in their name, nor do you name your files with two extensions (like `bar.js.ts`), then this extra cost probably isn't worth it, and you can use a much simpler check using the [`no-restricted-syntax`](https://eslint.org/docs/latest/rules/no-restricted-syntax) lint rule. + +The below config is several orders of magnitude faster than `import/extensions` as it does not do disk lookups, however it will false-positive on cases like the aforementioned `foo.js` module. + +```js +function banImportExtension(extension) { + const message = `Unexpected use of file extension (.${extension}) in import`; + const literalAttributeMatcher = `Literal[value=/\\.${extension}$/]`; + return [ + { + // import foo from 'bar.js'; + selector: `ImportDeclaration > ${literalAttributeMatcher}.source`, + message, + }, + { + // const foo = import('bar.js'); + selector: `ImportExpression > ${literalAttributeMatcher}.source`, + message, + }, + { + // type Foo = typeof import('bar.js'); + selector: `TSImportType > TSLiteralType > ${literalAttributeMatcher}`, + message, + }, + { + // const foo = require('foo.js'); + selector: `CallExpression[callee.name = "require"] > ${literalAttributeMatcher}.arguments`, + message, + }, + ]; +} + +module.exports = { + // ... other config ... + rules: { + 'no-restricted-syntax': [ + 'error', + ...banImportExtension('js'), + ...banImportExtension('jsx'), + ...banImportExtension('ts'), + ...banImportExtension('tsx'), + ], + }, +}; +``` + ### The `indent` / `@typescript-eslint/indent` rules This rule helps ensure your codebase follows a consistent indentation pattern. From 060c5f528ede32d4e8d118731276a96772af3928 Mon Sep 17 00:00:00 2001 From: Brad Zacher Date: Wed, 11 Jan 2023 11:46:57 +1030 Subject: [PATCH 2/3] move performance troubleshooting to its own page --- docs/linting/Troubleshooting.md | 134 +---------------- docs/linting/troubleshooting/FORMATTING.md | 37 +++-- docs/linting/troubleshooting/Performance.md | 150 ++++++++++++++++++++ packages/website/sidebars/sidebar.base.js | 1 + 4 files changed, 174 insertions(+), 148 deletions(-) create mode 100644 docs/linting/troubleshooting/Performance.md diff --git a/docs/linting/Troubleshooting.md b/docs/linting/Troubleshooting.md index 91fb816021bb..67e4316608f2 100644 --- a/docs/linting/Troubleshooting.md +++ b/docs/linting/Troubleshooting.md @@ -248,136 +248,4 @@ Rules such as [`no-unsafe-argument`](https://typescript-eslint.io/rules/no-unsaf ## My linting feels really slow -As mentioned in the [type-aware linting doc](./Typed_Linting.md), if you're using type-aware linting, your lint times should be roughly the same as your build times. - -If you're experiencing times much slower than that, then there are a few common culprits. - -### Wide includes in your `tsconfig` - -When using type-aware linting, you provide us with one or more tsconfigs. -We then will pre-parse all files so that full and complete type information is available. - -If you provide very wide globs in your `include` (such as `**/*`), it can cause many more files than you expect to be included in this pre-parse. -Additionally, if you provide no `include` in your tsconfig, then it is the same as providing the widest glob. - -Wide globs can cause TypeScript to parse things like build artifacts, which can heavily impact performance. -Always ensure you provide globs targeted at the folders you are specifically wanting to lint. - -### Wide includes in your ESLint options - -Specifying `tsconfig.json` paths in your ESLint commands is also likely to cause much more disk IO than expected. -Instead of globs that use `**` to recursively check all folders, prefer paths that use a single `*` at a time. - -```diff -- eslint --parser-options project:./**/tsconfig.json -+ eslint --parser-options project:./packages/*/tsconfig.json -``` - -See [Glob pattern in parser's option "project" slows down linting](https://github.com/typescript-eslint/typescript-eslint/issues/2611) for more details. - -### `eslint-plugin-prettier` - -This plugin surfaces prettier formatting problems at lint time, helping to ensure your code is always formatted. -However this comes at a quite a large cost - in order to figure out if there is a difference, it has to do a prettier format on every file being linted. -This means that each file will be parsed twice - once by ESLint, and once by Prettier. -This can add up for large codebases. - -Instead of using this plugin, we recommend using prettier's `--list-different` flag to detect if a file has not been correctly formatted. -For example, our CI is setup to run the following command automatically, which blocks PRs that have not been formatted: - -```bash npm2yarn -npm run prettier --list-different \"./**/*.{ts,mts,cts,tsx,js,mjs,cjs,jsx,json,md,css}\" -``` - -### `eslint-plugin-import` - -This is another great plugin that we use ourselves in this project. -However there are a few rules which can cause your lints to be really slow, because they cause the plugin to do its own parsing, and file tracking. -This double parsing adds up for large codebases. - -There are many rules that do single file static analysis, but we provide the following recommendations. - -We recommend you do not use the following rules, as TypeScript provides the same checks as part of standard type checking: - -- `import/named` -- `import/namespace` -- `import/default` -- `import/no-named-as-default-member` - -The following rules do not have equivalent checks in TypeScript, so we recommend that you only run them at CI/push time, to lessen the local performance burden. - -- `import/no-named-as-default` -- `import/no-cycle` -- `import/no-unused-modules` -- `import/no-deprecated` - -#### `import/extensions` - -There are two uses for this rule: - -1. to enforce file extensions are always used, -2. to enforce file extensions are never used. - -##### Enforcing extensions are used - -If you want to enforce file extensions are always used and you're **NOT** using `moduleResolution` `node16` or `nodenext`, then there's not really a good alternative for you, and you should continue using the `import/extensions` lint rule. - -If you want to enforce file extensions are always used and you **ARE** using `moduleResolution` `node16` or `nodenext`, then you don't need to use the lint rule at all because TypeScript will automatically enforce that you include extensions (TS 2834 & 2835)! - -##### Enforcing extensions are not used - -On the surface `import/extensions` seems like it should be fast for this use case, however the rule isn't just a pure AST-check - it has to resolve modules on disk so that it doesn't false positive on cases where you are importing modules with an extension as part of their name (eg `foo.js` resolves to `node_modules/foo.js/index.js`, so the `.js` is required). This disk lookup is costly and thus makes the rule slow. - -If your project doesn't use any `npm` packages with a file extension in their name, nor do you name your files with two extensions (like `bar.js.ts`), then this extra cost probably isn't worth it, and you can use a much simpler check using the [`no-restricted-syntax`](https://eslint.org/docs/latest/rules/no-restricted-syntax) lint rule. - -The below config is several orders of magnitude faster than `import/extensions` as it does not do disk lookups, however it will false-positive on cases like the aforementioned `foo.js` module. - -```js -function banImportExtension(extension) { - const message = `Unexpected use of file extension (.${extension}) in import`; - const literalAttributeMatcher = `Literal[value=/\\.${extension}$/]`; - return [ - { - // import foo from 'bar.js'; - selector: `ImportDeclaration > ${literalAttributeMatcher}.source`, - message, - }, - { - // const foo = import('bar.js'); - selector: `ImportExpression > ${literalAttributeMatcher}.source`, - message, - }, - { - // type Foo = typeof import('bar.js'); - selector: `TSImportType > TSLiteralType > ${literalAttributeMatcher}`, - message, - }, - { - // const foo = require('foo.js'); - selector: `CallExpression[callee.name = "require"] > ${literalAttributeMatcher}.arguments`, - message, - }, - ]; -} - -module.exports = { - // ... other config ... - rules: { - 'no-restricted-syntax': [ - 'error', - ...banImportExtension('js'), - ...banImportExtension('jsx'), - ...banImportExtension('ts'), - ...banImportExtension('tsx'), - ], - }, -}; -``` - -### The `indent` / `@typescript-eslint/indent` rules - -This rule helps ensure your codebase follows a consistent indentation pattern. -However this involves a _lot_ of computations across every single token in a file. -Across a large codebase, these can add up, and severely impact performance. - -We recommend not using this rule, and instead using a tool like [`prettier`](https://www.npmjs.com/package/prettier) to enforce a standardized formatting. +If you think you're having issues with performance, see our [Performance Troubleshooting documentation](./troubleshooting/Performance.md). diff --git a/docs/linting/troubleshooting/FORMATTING.md b/docs/linting/troubleshooting/FORMATTING.md index 47ff57c3b45c..213372f3bca8 100644 --- a/docs/linting/troubleshooting/FORMATTING.md +++ b/docs/linting/troubleshooting/FORMATTING.md @@ -6,6 +6,17 @@ title: What About Formatting? We strongly recommend against using ESLint for formatting. We strongly recommend using [Prettier](https://prettier.io), [dprint](https://dprint.dev), or an equivalent instead. +## ESLint Core and Formatting + +Per [ESLint's 2020 Changes to Rule Policies blog post](https://eslint.org/blog/2020/05/changes-to-rules-policies#what-are-the-changes): + +> Stylistic rules are frozen - we won't be adding any more options to stylistic rules. +> We've learned that there's no way to satisfy everyone's personal preferences, and most of the rules already have a lot of difficult-to-understand options. +> Stylistic rules are those related to spacing, conventions, and generally anything that does not highlight an error or a better way to do something. + +We support the ESLint team's decision and backing logic to move away from stylistic rules. +With the exception of bug fixes, no new formatting-related pull requests will be accepted into typescript-eslint. + ## Formatters vs. Linters **Formatters** are tools that verify and correct whitespace issues in code, such as spacing and newlines. @@ -16,13 +27,18 @@ Linters often take seconds or more to run because they apply many logical rules ### Problems with Using Linters as Formatters -Linters apply much more work than formatters -- often including potentially multiple rounds of rule fixers. -That generally makes them run orders of magnitude slower. +Linters are designed to run in a parse, check, report, fix cycle. This means that there is a lot of intermediate work that needs to be done before a linter can fix a formatting issue with your code. + +Additionally linters typically run each rule isolated from one another. This has several problems with it like: -Additionally, modern formatters such as Prettier are architected in a way that applies formatting to all code regardless of original formatting. -Linters typically run on a rule-by-rule basis, typically resulting in many edge cases and missed coverage in formatting. +- any two lint rules can't share config meaning one lint rule's fixer might introduce a violation of another lint rule's fixer (eg one lint rule might use the incorrect indentation character). +- lint rule fixers can conflict (apply to the same code range), forcing the linter to perform an additional cycle to attempt to apply a fixer to a clean set of code. -### Suggested Usage +These problems cause a linter to be much slower and, more importantly, much less consistent and less able to handle edge-cases than a purpose-built formatter. + +Modern formatters such as Prettier are architected in a way that applies formatting to all code regardless of original formatting which helps them be more consistent. + +### Suggested Usage - Prettier We recommend using [`eslint-config-prettier`](https://github.com/prettier/eslint-config-prettier) to disable formatting rules in your ESLint configuration. You can then configure your formatter separately from ESLint. @@ -43,13 +59,4 @@ module.exports = { }; ``` -## ESLint Core and Formatting - -Per [ESLint's 2020 Changes to Rule Policies blog post](https://eslint.org/blog/2020/05/changes-to-rules-policies#what-are-the-changes): - -> Stylistic rules are frozen - we won't be adding any more options to stylistic rules. -> We've learned that there's no way to satisfy everyone's personal preferences, and most of the rules already have a lot of difficult-to-understand options. -> Stylistic rules are those related to spacing, conventions, and generally anything that does not highlight an error or a better way to do something. - -We support the ESLint team's decision and backing logic to move away from stylistic rules. -With the exception of bug fixes, no new formatting-related pull requests will be accepted into typescript-eslint. +Note that even if you don't use `prettier`, you can use `eslint-config-prettier` as it exclusively turns **off** all formatting rules. diff --git a/docs/linting/troubleshooting/Performance.md b/docs/linting/troubleshooting/Performance.md new file mode 100644 index 000000000000..6f696a7a05f0 --- /dev/null +++ b/docs/linting/troubleshooting/Performance.md @@ -0,0 +1,150 @@ +--- +id: performance-troubleshooting +title: Performance Troubleshooting +--- + +As mentioned in the [type-aware linting doc](../Typed_Linting.md), if you're using type-aware linting, your lint times should be roughly the same as your build times. + +If you're experiencing times much slower than that, then there are a few common culprits. + +## Wide includes in your `tsconfig` + +When using type-aware linting, you provide us with one or more tsconfigs. +We then will pre-parse all files so that full and complete type information is available. + +If you provide very wide globs in your `include` (such as `**/*`), it can cause many more files than you expect to be included in this pre-parse. +Additionally, if you provide no `include` in your tsconfig, then it is the same as providing the widest glob. + +Wide globs can cause TypeScript to parse things like build artifacts, which can heavily impact performance. +Always ensure you provide globs targeted at the folders you are specifically wanting to lint. + +## Wide includes in your ESLint options + +Specifying `tsconfig.json` paths in your ESLint commands is also likely to cause much more disk IO than expected. +Instead of globs that use `**` to recursively check all folders, prefer paths that use a single `*` at a time. + +```js title=".eslintrc.js" +module.exports = { + extends: [ + 'eslint:recommended', + 'plugin:@typescript-eslint/recommended', + 'plugin:@typescript-eslint/recommended-requiring-type-checking', + ], + parser: '@typescript-eslint/parser', + parserOptions: { + tsconfigRootDir: __dirname, + // Remove this line + project: ['./**/tsconfig.json'], + // Add this line + project: ['./packages/*/tsconfig.json'], + }, + plugins: ['@typescript-eslint'], + root: true, +}; +``` + +See [Glob pattern in parser's option "project" slows down linting](https://github.com/typescript-eslint/typescript-eslint/issues/2611) for more details. + +## The `indent` / `@typescript-eslint/indent` rules + +This rule helps ensure your codebase follows a consistent indentation pattern. +However this involves a _lot_ of computations across every single token in a file. +Across a large codebase, these can add up, and severely impact performance. + +We recommend not using this rule, and instead using a tool like [`prettier`](https://www.npmjs.com/package/prettier) to enforce a standardized formatting. + +See our [documentation on formatting](./Formatting.md) for more information. + +## `eslint-plugin-prettier` + +This plugin surfaces prettier formatting problems at lint time, helping to ensure your code is always formatted. +However this comes at a quite a large cost - in order to figure out if there is a difference, it has to do a prettier format on every file being linted. +This means that each file will be parsed twice - once by ESLint, and once by Prettier. +This can add up for large codebases. + +Instead of using this plugin, we recommend using prettier's `--list-different` flag to detect if a file has not been correctly formatted. +For example, our CI is setup to run the following command automatically, which blocks PRs that have not been formatted: + +```bash npm2yarn +npm run prettier --list-different \"./**/*.{ts,mts,cts,tsx,js,mjs,cjs,jsx,json,md,css}\" +``` + +## `eslint-plugin-import` + +This is another great plugin that we use ourselves in this project. +However there are a few rules which can cause your lints to be really slow, because they cause the plugin to do its own parsing, and file tracking. +This double parsing adds up for large codebases. + +There are many rules that do single file static analysis, but we provide the following recommendations. + +We recommend you do not use the following rules, as TypeScript provides the same checks as part of standard type checking: + +- `import/named` +- `import/namespace` +- `import/default` +- `import/no-named-as-default-member` + +The following rules do not have equivalent checks in TypeScript, so we recommend that you only run them at CI/push time, to lessen the local performance burden. + +- `import/no-named-as-default` +- `import/no-cycle` +- `import/no-unused-modules` +- `import/no-deprecated` + +### `import/extensions` + +#### Enforcing extensions are used + +If you want to enforce file extensions are always used and you're **NOT** using `moduleResolution` `node16` or `nodenext`, then there's not really a good alternative for you, and you should continue using the `import/extensions` lint rule. + +If you want to enforce file extensions are always used and you **ARE** using `moduleResolution` `node16` or `nodenext`, then you don't need to use the lint rule at all because TypeScript will automatically enforce that you include extensions! + +#### Enforcing extensions are not used + +On the surface `import/extensions` seems like it should be fast for this use case, however the rule isn't just a pure AST-check - it has to resolve modules on disk so that it doesn't false positive on cases where you are importing modules with an extension as part of their name (eg `foo.js` resolves to `node_modules/foo.js/index.js`, so the `.js` is required). This disk lookup is costly and thus makes the rule slow. + +If your project doesn't use any `npm` packages with a file extension in their name, nor do you name your files with two extensions (like `bar.js.ts`), then this extra cost probably isn't worth it, and you can use a much simpler check using the [`no-restricted-syntax`](https://eslint.org/docs/latest/rules/no-restricted-syntax) lint rule. + +The below config is several orders of magnitude faster than `import/extensions` as it does not do disk lookups, however it will false-positive on cases like the aforementioned `foo.js` module. + +```js +function banImportExtension(extension) { + const message = `Unexpected use of file extension (.${extension}) in import`; + const literalAttributeMatcher = `Literal[value=/\\.${extension}$/]`; + return [ + { + // import foo from 'bar.js'; + selector: `ImportDeclaration > ${literalAttributeMatcher}.source`, + message, + }, + { + // const foo = import('bar.js'); + selector: `ImportExpression > ${literalAttributeMatcher}.source`, + message, + }, + { + // type Foo = typeof import('bar.js'); + selector: `TSImportType > TSLiteralType > ${literalAttributeMatcher}`, + message, + }, + { + // const foo = require('foo.js'); + selector: `CallExpression[callee.name = "require"] > ${literalAttributeMatcher}.arguments`, + message, + }, + ]; +} + +module.exports = { + // ... other config ... + rules: { + 'no-restricted-syntax': [ + 'error', + ...banImportExtension('js'), + ...banImportExtension('jsx'), + ...banImportExtension('ts'), + ...banImportExtension('tsx'), + ], + }, +}; +``` diff --git a/packages/website/sidebars/sidebar.base.js b/packages/website/sidebars/sidebar.base.js index fe84b9c1ae39..e6e7401727f1 100644 --- a/packages/website/sidebars/sidebar.base.js +++ b/packages/website/sidebars/sidebar.base.js @@ -23,6 +23,7 @@ module.exports = { collapsible: false, type: 'category', items: [ + 'linting/troubleshooting/performance-troubleshooting', 'linting/troubleshooting/formatting', 'linting/troubleshooting/tslint', ], From 6756ff31bb3800b840778d2ed8209971f1b3f7de Mon Sep 17 00:00:00 2001 From: Brad Zacher Date: Thu, 12 Jan 2023 12:05:39 +1030 Subject: [PATCH 3/3] review --- docs/linting/troubleshooting/{FORMATTING.md => Formatting.md} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename docs/linting/troubleshooting/{FORMATTING.md => Formatting.md} (98%) diff --git a/docs/linting/troubleshooting/FORMATTING.md b/docs/linting/troubleshooting/Formatting.md similarity index 98% rename from docs/linting/troubleshooting/FORMATTING.md rename to docs/linting/troubleshooting/Formatting.md index 213372f3bca8..14c9d102dd00 100644 --- a/docs/linting/troubleshooting/FORMATTING.md +++ b/docs/linting/troubleshooting/Formatting.md @@ -29,7 +29,7 @@ Linters often take seconds or more to run because they apply many logical rules Linters are designed to run in a parse, check, report, fix cycle. This means that there is a lot of intermediate work that needs to be done before a linter can fix a formatting issue with your code. -Additionally linters typically run each rule isolated from one another. This has several problems with it like: +Additionally linters typically run each rule isolated from one another. This has several problems with it such as: - any two lint rules can't share config meaning one lint rule's fixer might introduce a violation of another lint rule's fixer (eg one lint rule might use the incorrect indentation character). - lint rule fixers can conflict (apply to the same code range), forcing the linter to perform an additional cycle to attempt to apply a fixer to a clean set of code.