[labs/analyzer] Add template parser to analyzer #4267
Conversation
🦋 Changeset detectedLatest commit: 2ab8669 The changes in this PR will be included in the next version bump. This PR includes changesets to release 9 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
📊 Tachometer Benchmark ResultsSummarynop-update
render
update
update-reflect
Resultsthis-change
render
update
update-reflect
this-change, tip-of-tree, previous-release
render
update
nop-update
this-change, tip-of-tree, previous-release
render
update
this-change, tip-of-tree, previous-release
render
update
update-reflect
|
|
The size of lit-html.js and lit-core.min.js are as expected. |
| export const parseLitTemplate = ( | ||
| node: ts.TaggedTemplateExpression, | ||
| ts: TypeScript, | ||
| _checker: ts.TypeChecker |
There was a problem hiding this comment.
Worth removing as not used currently?
There was a problem hiding this comment.
I have it here for consistency with other functions that need the checker, because it's kind of unusual that this function doesn't need the checker. In case we need to use the checker in here in the future, it won't be a breaking change to public consumers.
| import type ts from 'typescript'; | ||
| import {_$LH} from 'lit-html/private-ssr-support.js'; | ||
| import {parseFragment} from 'parse5'; | ||
| // import type {Attribute} from 'parse5/dist/common/token.js'; |
There was a problem hiding this comment.
Intended to leave in?
| const values = ts.isNoSubstitutionTemplateLiteral(node.template) | ||
| ? [] | ||
| : node.template.templateSpans.map((s) => s.expression); | ||
| const parts: Array<PartInfo> = []; |
There was a problem hiding this comment.
It might be good to clarify the use cases for the API's here. i.e. whether we intend the DOM AST that comes out to be mutable or not, since the parts list will only be the "as-parsed" values.
This could either be made a getter that reifies the parts list each access (slower), or could add an "invalidateParts" API?
And are you planning on adding something to go from a LitTemplate back to a ts.TaggedTemplateExpression?
There was a problem hiding this comment.
It's obv fine for some use cases, where you just want to statically analyze only, but for others the parts sort of seems not that useful.
There was a problem hiding this comment.
fwiw in the lint side of things, we don't need to mutate nodes but we do need a way to translate between nodes or positions. i.e. we need a way to go for a parse5 node to an estree node (whether by reference or just by knowing the position)
There was a problem hiding this comment.
Adding more in the comment. I personally view this utility right now as being similar to asking a parser to parse a file - it'll generally give you a new AST each time you call it.
I think this can be a primitive used by a cache-backed function with a different name. Like getLitTemplate(node) that assumes the TS node is immutable and will return the same Lit template instance on repeated calls.
48032bd to
b4c6ae2
Compare
ff05c9c to
21a62dd
Compare
a853ce1 to
ec459b9
Compare
ec459b9 to
547fd74
Compare
| * Returns true if the decorator site is a simple called decorator factory of | ||
| * the form `@decoratorName()`. |
There was a problem hiding this comment.
| * Returns true if the decorator site is a simple called decorator factory of | |
| * the form `@decoratorName()`. | |
| * Given a decorator name, returns true if the decorator site is a simple called | |
| * decorator factory of the form `@decoratorName()` where `decoratorName` | |
| * matches the given name. |
There was a problem hiding this comment.
This was a file move that Git or GitHub didn't pick up. I'd rather keep this PR limited to the template parser.
| /** | ||
| * A narrower type for ts.Decorator that represents the shape of an analyzable | ||
| * `@customElement('x')` callsite. | ||
| */ | ||
| interface PropertyDecorator extends ts.Decorator { | ||
| readonly expression: ts.CallExpression; | ||
| } |
There was a problem hiding this comment.
Could also remove this and rename CustomElementDecorator to something like SimpleDecorator. Also, jsdoc on this should be updated to say @property()
There was a problem hiding this comment.
Same as the other comment about this being a file move.
| if (!ts.isTaggedTemplateExpression(node)) { | ||
| return false; | ||
| } | ||
| if (ts.isIdentifier(node.tag)) { |
There was a problem hiding this comment.
We can also maybe cover a lot of edge stuff with supporting comments with something maybe like:
node.getFullText().slice(node.getFullStart(), node.getStart()).includes('@litHtmlTemplate')
but more idomatic comment gathering
| : node.template.templateSpans; | ||
|
|
||
| const parts: Array<PartInfo> = []; | ||
| const [html, boundAttributeNames] = getTemplateHtml(strings, 1); |
There was a problem hiding this comment.
OMG this is a clever and wild approach! I was incredibly confused as to why there were litPart comment nodes in a lit tagged template literal and now this entire file makes sense now!
There was a problem hiding this comment.
Thanks! This was the plan from a long time ago :) I hope it hold up actually.
| // in the prepared and parsed HTML. | ||
|
|
||
| // TODO (justinfagnani): implement line and column adjustments | ||
| let lineAdjust = 0; |
There was a problem hiding this comment.
Do we really even need lines? The TS AST nor many ESTree-compatible ASTs use a start and end offset. The exception seems to be babel. I'm in the camp that we can either wipe them or just ignore them for the sake of simplicity and performance
There was a problem hiding this comment.
parse5 includes lines, so I didn't want to leave them unadjusted and incorrect.
| sourceCodeLocationInfo: true, | ||
| }); | ||
|
|
||
| traverse(ast, { |
There was a problem hiding this comment.
Why not do some pre-processing of the template strings before traversing. e.g.
- Create a map of original-positions to processed-positions and vice versa
- iterate through the strings char by char
- store index of EVERY char in both maps
- When we get to the end of the string and to an expression replace the expression with a comment
- original position is += the full text length of the expression
- processed position is the length of your marker text
Now, if you have a map of all positions mapped from one template to another, when you traverse the parse5 tree and visit a node, it doesn't matter much what the node is, as long as it has a sourceCodeLocation. The only times you'd have to care is if it's an element. In this case you'd have to also update el.sourceCodeLocation.{attrs,startTag,endTag}
The negatives here are that you are making a map that is memory size of O(n) where n is the max length of the template literal but it should be a map of Map<number,number>, so how bad can it be???
There was a problem hiding this comment.
I don't fully follow this approach. Can you explain it to me on a call?
| type TypeScript = typeof ts; | ||
|
|
||
| // Why, oh why are parse5 types so weird? We re-export them to make them easier | ||
| // to use. |
There was a problem hiding this comment.
you should be able to pull these from @parse5/tools itself now
i think it re-exports the defaults
There was a problem hiding this comment.
I don't see where to get all of these types from.
08d8c28 to
4ce888c
Compare
ed676f7 to
6318a70
Compare
| '@lit-labs/analyzer': minor | ||
| --- | ||
|
|
||
| Add a template parser |
| "test:server:uvu": { | ||
| "#comment": "The quotes around the file regex must be double quotes on windows!", | ||
| "command": "uvu test/server \"_test\\.js$\" -i \"lit-html/\"", | ||
| "command": "uvu test/server \"_test\\.js$\" -i \"lit/template_test\"", |
There was a problem hiding this comment.
What's with these changes?
There was a problem hiding this comment.
Just getting the uvu tests run by uvu and the node tests to be run run by node. I want to convert everything to node soon.
Builds on #4261
Adds a
parseLitTemplate()function tolib/lit-html/template.ts. This returns an extension of a parse5 DocumentFragment that contains parts and strings, and where some nodes like comments and attributes are annotated with Lit parts. The parts have references to their expressions to allow easy traversal into nested expressions and templates.I didn't yet port over the similar code in SSR or the compiler. These might have some different requirements (like parallel traversal or re-writing comment markers). Other future extensions could be detecting common invalid binding locations for linting.