[labs/ssr] fix server templates dropping top level `td` elements #4519

AndrewJakubowicz · 2024-01-25T00:33:03Z

Context

Using parse from parse5 everywhere for server templates is not ideal, and unfortunately has consequences for some cases such as the issue template containing a top level td element. Parsing this as a page level HTML causes the element to be removed (which matches browser behavior). However, we instead want to treat this case as a fragment.

Fix

This is a possible fix. I'm not super stoked about it but it does improve the situation.

Basically, there are 3 tags that I could find that parseFragment will not handle as a top-level element: html, head, and body.
Thus, detect those cases and only use parse if one of these top-level tags is detected. Otherwise use parseFragment.

This change also strips off comment nodes.

Test plan

Added a regression test for the template flagged in the issue.

…f parseFragment

changeset-bot · 2024-01-25T00:33:06Z

🦋 Changeset detected

Latest commit: 2c55b58

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@lit-labs/ssr	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

github-actions · 2024-01-25T00:33:21Z

📊 Tachometer Benchmark Results

Summary

nop-update

this-change, tip-of-tree, previous-release: unsure 🔍 -6% - +2% (-0.71ms - +0.25ms)
^{this-change vs tip-of-tree}

render

this-change: 51.03ms - 53.66ms
this-change, tip-of-tree, previous-release: unsure 🔍 -3% - +4% (-0.64ms - +0.73ms)
^{this-change vs tip-of-tree}
this-change, tip-of-tree, previous-release: unsure 🔍 -3% - +2% (-0.97ms - +0.58ms)
^{this-change vs tip-of-tree}
this-change, tip-of-tree, previous-release: unsure 🔍 -4% - +1% (-1.38ms - +0.36ms)
^{this-change vs tip-of-tree}

update

this-change: 571.17ms - 583.01ms
this-change, tip-of-tree, previous-release: unsure 🔍 -7% - +6% (-3.10ms - +2.68ms)
^{this-change vs tip-of-tree}
this-change, tip-of-tree, previous-release: unsure 🔍 -2% - +3% (-1.42ms - +2.49ms)
^{this-change vs tip-of-tree}
this-change, tip-of-tree, previous-release: unsure 🔍 -2% - +1% (-9.13ms - +5.52ms)
^{this-change vs tip-of-tree}

update-reflect

this-change: 569.76ms - 575.96ms
this-change, tip-of-tree, previous-release: unsure 🔍 -1% - +2% (-4.00ms - +9.52ms)
^{this-change vs tip-of-tree}

Results

this-change

Browser: chrome-headless 120.0.6099.224
Sample size: 50
Built by: Benchmarks #6055
Commit: 2c55b58

render

Version	Avg time	vs
	51.03ms - 53.66ms	-

update

Version	Avg time	vs
	571.17ms - 583.01ms	-

update-reflect

Version	Avg time	vs
	569.76ms - 575.96ms	-

this-change, tip-of-tree, previous-release

Browser: chrome-headless 120.0.6099.224
Sample size: 30
Built by: Benchmarks #6055
Commit: 2c55b58

render

Version	Avg time	vs this-change	vs tip-of-tree tip-of-tree	vs previous-release previous-release
this-change	18.67ms - 19.66ms	-	unsure 🔍 -3% - +4% -0.64ms - +0.73ms	unsure 🔍 -7% - +0% -1.40ms - +0.09ms
tip-of-tree tip-of-tree	18.64ms - 19.60ms	unsure 🔍 -4% - +3% -0.73ms - +0.64ms	-	unsure 🔍 -7% - +0% -1.43ms - +0.03ms
previous-release previous-release	19.26ms - 20.37ms	unsure 🔍 -1% - +7% -0.09ms - +1.40ms	unsure 🔍 -0% - +8% -0.03ms - +1.43ms	-

update

Version	Avg time	vs this-change	vs tip-of-tree tip-of-tree	vs previous-release previous-release
this-change	41.26ms - 44.89ms	-	unsure 🔍 -7% - +6% -3.10ms - +2.68ms	faster ✔ 3% - 13% 1.16ms - 6.23ms
tip-of-tree tip-of-tree	41.04ms - 45.53ms	unsure 🔍 -6% - +7% -2.68ms - +3.10ms	-	faster ✔ 2% - 13% 0.63ms - 6.34ms
previous-release previous-release	45.01ms - 48.53ms	slower ❌ 2% - 15% 1.16ms - 6.23ms	slower ❌ 1% - 15% 0.63ms - 6.34ms	-

nop-update

Version	Avg time	vs this-change	vs tip-of-tree tip-of-tree	vs previous-release previous-release
this-change	11.36ms - 12.08ms	-	unsure 🔍 -6% - +2% -0.71ms - +0.25ms	faster ✔ 0% - 8% 0.00ms - 1.04ms
tip-of-tree tip-of-tree	11.62ms - 12.28ms	unsure 🔍 -2% - +6% -0.25ms - +0.71ms	-	unsure 🔍 -6% - +2% -0.79ms - +0.21ms
previous-release previous-release	11.86ms - 12.62ms	unsure 🔍 -0% - +9% +0.00ms - +1.04ms	unsure 🔍 -2% - +7% -0.21ms - +0.79ms	-

this-change, tip-of-tree, previous-release

Browser: chrome-headless 120.0.6099.224
Sample size: 50
Built by: Benchmarks #6055
Commit: 2c55b58

render

Version	Avg time	vs this-change	vs tip-of-tree tip-of-tree	vs previous-release previous-release
this-change	34.70ms - 35.80ms	-	unsure 🔍 -3% - +2% -0.97ms - +0.58ms	unsure 🔍 -3% - +1% -1.12ms - +0.49ms
tip-of-tree tip-of-tree	34.91ms - 35.99ms	unsure 🔍 -2% - +3% -0.58ms - +0.97ms	-	unsure 🔍 -3% - +2% -0.92ms - +0.68ms
previous-release previous-release	34.98ms - 36.15ms	unsure 🔍 -1% - +3% -0.49ms - +1.12ms	unsure 🔍 -2% - +3% -0.68ms - +0.92ms	-

update

Version	Avg time	vs this-change	vs tip-of-tree tip-of-tree	vs previous-release previous-release
this-change	78.84ms - 81.63ms	-	unsure 🔍 -2% - +3% -1.42ms - +2.49ms	unsure 🔍 -5% - +2% -3.86ms - +1.73ms
tip-of-tree tip-of-tree	78.33ms - 81.07ms	unsure 🔍 -3% - +2% -2.49ms - +1.42ms	-	unsure 🔍 -5% - +1% -4.38ms - +1.18ms
previous-release previous-release	78.88ms - 83.72ms	unsure 🔍 -2% - +5% -1.73ms - +3.86ms	unsure 🔍 -2% - +6% -1.18ms - +4.38ms	-

this-change, tip-of-tree, previous-release

Browser: chrome-headless 120.0.6099.224
Sample size: 50
Built by: Benchmarks #6055
Commit: 2c55b58

render

Version	Avg time	vs this-change	vs tip-of-tree tip-of-tree	vs previous-release previous-release
this-change	36.03ms - 36.95ms	-	unsure 🔍 -4% - +1% -1.38ms - +0.36ms	unsure 🔍 -2% - +1% -0.88ms - +0.52ms
tip-of-tree tip-of-tree	36.27ms - 37.73ms	unsure 🔍 -1% - +4% -0.36ms - +1.38ms	-	unsure 🔍 -2% - +3% -0.57ms - +1.23ms
previous-release previous-release	36.14ms - 37.20ms	unsure 🔍 -1% - +2% -0.52ms - +0.88ms	unsure 🔍 -3% - +2% -1.23ms - +0.57ms	-

update

Version	Avg time	vs this-change	vs tip-of-tree tip-of-tree	vs previous-release previous-release
this-change	595.53ms - 605.22ms	-	unsure 🔍 -2% - +1% -9.13ms - +5.52ms	unsure 🔍 -1% - +1% -6.19ms - +7.30ms
tip-of-tree tip-of-tree	596.69ms - 607.67ms	unsure 🔍 -1% - +2% -5.52ms - +9.13ms	-	unsure 🔍 -1% - +2% -4.86ms - +9.59ms
previous-release previous-release	595.13ms - 604.51ms	unsure 🔍 -1% - +1% -7.30ms - +6.19ms	unsure 🔍 -2% - +1% -9.59ms - +4.86ms	-

update-reflect

Version	Avg time	vs this-change	vs tip-of-tree tip-of-tree	vs previous-release previous-release
this-change	600.66ms - 610.38ms	-	unsure 🔍 -1% - +2% -4.00ms - +9.52ms	unsure 🔍 -1% - +1% -4.78ms - +7.45ms
tip-of-tree tip-of-tree	598.06ms - 607.46ms	unsure 🔍 -2% - +1% -9.52ms - +4.00ms	-	unsure 🔍 -1% - +1% -7.42ms - +4.56ms
previous-release previous-release	600.48ms - 607.90ms	unsure 🔍 -1% - +1% -7.45ms - +4.78ms	unsure 🔍 -1% - +1% -4.56ms - +7.42ms	-

_{tachometer-reporter-action v2 for Benchmarks}

github-actions · 2024-01-25T00:34:49Z

The size of lit-html.js and lit-core.min.js are as expected.

justinfagnani · 2024-01-25T00:57:13Z

I suspect there's a bit more we can do here with more advanced parts of the parse5 API. Looking at the implementation of parseFragment() it seems like we can get at the root node and see attributes on it:

import {Parser} from 'parse5';

const parser = Parser.getFragmentParser();
parser.tokenizer.write('<html foo><div></div></html>', true);

// This has the `foo` attribute
console.log(parser.treeAdapter.getFirstChild(parser.document));

// This only has the <div>
console.log(parser.getFragment());

I'm not sure yet how to tie this into a fix, but it seems like more of the information is there internally.

AndrewJakubowicz · 2024-01-25T02:35:02Z

I agree Justin! I'll take a look a little more deeply into what parse5 gives us tomorrow.

AndrewJakubowicz · 2024-01-25T22:04:27Z

@justinfagnani I think an inexpensive regex approach is probably going to be the best to capture the couple page level elements. Using parseFragment specifically on templates containing a top level <html> doesn't work well (even if manually rigged up) because parse5 removes the sourceCodeLocation. I haven't found a way to populate the source code location, which is important because we use it to flushTo and skipTo around Lit markers.

augustjk

Unless we unship the parse() change to regress on the page level element support, I like this is a strict improvement of what we currently have.

Trying to not use parse() all together could be explored on top of this change too.

justinfagnani · 2024-01-25T22:36:20Z

packages/labs/ssr/src/lib/render-value.ts

+ * we use a page-level template `parse`.
+ */
+const REGEXP_TEMPLATE_STARTS_WITH_PAGE_TAG =
+  /^\s*(<(!doctype|html|head|body))/i;


Great suggestion. I've updated regex and added a bunch of tests. Things look pretty great with the comment regex addition.

justinfagnani · 2024-01-25T22:36:54Z

@AndrewJakubowicz thanks for looking into it!

I think the limitations of using non-fragment should be documented along with the tag. I modified the regex to handle comments (I think) so maybe there wouldn't be any to mention.

packages/labs/ssr/src/lib/render-value.ts

augustjk

Thank you!

packages/labs/ssr/src/lib/render-value.ts

.changeset/tough-crews-cover.md

Co-authored-by: Augustine Kim <[email protected]>

AndrewJakubowicz added 2 commits January 24, 2024 16:26

be much more deliberate and specific for when parse is used instead o…

d120629

…f parseFragment

add changeset

2321e27

AndrewJakubowicz requested a review from kevinpschaaf as a code owner January 25, 2024 00:33

AndrewJakubowicz requested a review from justinfagnani January 25, 2024 00:33

AndrewJakubowicz requested a review from augustjk January 25, 2024 00:34

remove some logs

cf16adb

AndrewJakubowicz removed request for augustjk, justinfagnani and kevinpschaaf January 25, 2024 02:46

augustjk approved these changes Jan 25, 2024

View reviewed changes

AndrewJakubowicz requested review from justinfagnani and rictic January 25, 2024 22:16

justinfagnani reviewed Jan 25, 2024

View reviewed changes

add comment handling and a whole lot of tests

bcc126d

github-advanced-security bot found potential problems Jan 25, 2024

View reviewed changes

packages/labs/ssr/src/lib/render-value.ts Fixed Show fixed Hide fixed

packages/labs/ssr/src/lib/render-value.ts Fixed Show fixed Hide fixed

fix exponential backtracking inefficiency

81ab4c5

github-advanced-security bot found potential problems Jan 25, 2024

View reviewed changes

packages/labs/ssr/src/lib/render-value.ts Fixed Show fixed Hide fixed

AndrewJakubowicz added 2 commits January 25, 2024 15:32

clean up regex further

df65710

even more robustness, the regex should only run on server templates

a709efa

AndrewJakubowicz requested review from augustjk and justinfagnani January 25, 2024 23:47

augustjk approved these changes Jan 25, 2024

View reviewed changes

packages/labs/ssr/src/lib/render-value.ts Outdated Show resolved Hide resolved

augustjk reviewed Jan 25, 2024

View reviewed changes

.changeset/tough-crews-cover.md Outdated Show resolved Hide resolved

AndrewJakubowicz and others added 2 commits January 25, 2024 16:05

remove isServerTemplate flag and use !hydratable

4214b96

Update .changeset/tough-crews-cover.md

2c55b58

Co-authored-by: Augustine Kim <[email protected]>

justinfagnani approved these changes Jan 27, 2024

View reviewed changes

AndrewJakubowicz merged commit 8d3e305 into main Jan 29, 2024

AndrewJakubowicz deleted the server-html-templates-td branch January 29, 2024 18:36

lit-robot mentioned this pull request Jan 31, 2024

Version Packages #4532

Merged

	/^\s*(<(!doctype\|html\|head\|body))/i;
	/^(\s\|<!--.-->)*(<(!doctype\|html\|head\|body))/is;

[labs/ssr] fix server templates dropping top level td elements #4519

[labs/ssr] fix server templates dropping top level td elements #4519

Uh oh!

Conversation

AndrewJakubowicz commented Jan 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Context

Fix

Test plan

Uh oh!

changeset-bot bot commented Jan 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

github-actions bot commented Jan 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📊 Tachometer Benchmark Results

Summary

nop-update

render

update

update-reflect

Results

render

update

update-reflect

render

update

nop-update

render

update

render

update

update-reflect

Uh oh!

github-actions bot commented Jan 25, 2024

Uh oh!

justinfagnani commented Jan 25, 2024

Uh oh!

AndrewJakubowicz commented Jan 25, 2024

Uh oh!

AndrewJakubowicz commented Jan 25, 2024

Uh oh!

augustjk left a comment

Choose a reason for hiding this comment

Uh oh!

justinfagnani Jan 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AndrewJakubowicz Jan 25, 2024

Choose a reason for hiding this comment

Uh oh!

justinfagnani commented Jan 25, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

augustjk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

[labs/ssr] fix server templates dropping top level `td` elements #4519

[labs/ssr] fix server templates dropping top level `td` elements #4519

AndrewJakubowicz commented Jan 25, 2024 •

edited

Loading

changeset-bot bot commented Jan 25, 2024 •

edited

Loading

github-actions bot commented Jan 25, 2024 •

edited

Loading

justinfagnani Jan 25, 2024 •

edited

Loading