[infra][lit-labs/compiler] lit-html tests are now also compiled and tested #4102

AndrewJakubowicz · 2023-08-15T21:21:23Z

Issue: #189
RFC: lit/rfcs#21
Full prototype PR: #3984

Context

This is an infrastructure only, and test-only change. No user facing impact.
It does change some lit-html tests, but only slightly such that they can test both uncompiled & compiled templates.

Specifically, any tests that explicitly tested detection of uncompiled templates, need to handle the whole file being compiled.

The compiler also strips markers from comments which required one test to be updated.

Features

A new tsconfig in lit-html which only targets the test/ directory.
A bespoke TypeScript emitter (in lit-labs/compiler) so we can compile the lit-html/test directory (and not run into a TS version issue if lit-html was housing the transform).
A new lit-html test command: test:dev:compiled.
An extra path is handled in the web-test-runner config so the compiled/ test path can be tested.

Test plan

This whole PR is a test plan ensuring robust tests for the lit-labs/compiler transform.
It proves that the compiler gracefully handles all the edge cases we test in the lit-html tests.

Future consideration

Note that in the future when the whole repo is on TS 5.x, we'll be able to use a rollup config to compile the tests instead of a script in the compiler package.

Thank you so much!

changeset-bot · 2023-08-15T21:21:37Z

🦋 Changeset detected

Latest commit: a888ebf

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 0 packages

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

github-actions · 2023-08-15T21:21:41Z

📊 Tachometer Benchmark Results

Summary

⏳ Benchmarks are currently running. Results below are out of date.

nop-update

this-change, tip-of-tree, previous-release: unsure 🔍 -1% - +14% (-0.08ms - +2.61ms)
^{this-change vs tip-of-tree}

render

this-change: 67.10ms - 69.97ms
this-change, tip-of-tree, previous-release: unsure 🔍 -1% - +6% (-0.33ms - +1.65ms)
^{this-change vs tip-of-tree}
this-change, tip-of-tree, previous-release: faster ✔ 1% - 3% (0.33ms - 1.72ms)
^{this-change vs tip-of-tree}
this-change, tip-of-tree, previous-release: unsure 🔍 -1% - +2% (-0.58ms - +0.95ms)
^{this-change vs tip-of-tree}

update

this-change: 665.51ms - 678.34ms
this-change, tip-of-tree, previous-release: unsure 🔍 -6% - +3% (-4.38ms - +1.94ms)
^{this-change vs tip-of-tree}
this-change, tip-of-tree, previous-release: unsure 🔍 -2% - +0% (-2.14ms - +0.50ms)
^{this-change vs tip-of-tree}
this-change, tip-of-tree, previous-release: unsure 🔍 -1% - +1% (-4.24ms - +5.32ms)
^{this-change vs tip-of-tree}

update-reflect

this-change: 642.90ms - 650.84ms
this-change, tip-of-tree, previous-release: unsure 🔍 -1% - +0% (-6.78ms - +0.33ms)
^{this-change vs tip-of-tree}

Results

⏳ Benchmarks are currently running. Results below are out of date.

this-change

Browser: chrome-headless 115.0.5790.170
Sample size: 50
Built by: Benchmarks #5239
Commit: 4cdfecd

render

Version	Avg time	vs
	67.10ms - 69.97ms	-

update

Version	Avg time	vs
	665.51ms - 678.34ms	-

update-reflect

Version	Avg time	vs
	642.90ms - 650.84ms	-

this-change, tip-of-tree, previous-release

Browser: chrome-headless 115.0.5790.170
Sample size: 30
Built by: Benchmarks #5239
Commit: 4cdfecd

render

Version	Avg time	vs this-change	vs tip-of-tree tip-of-tree	vs previous-release previous-release
this-change	27.74ms - 29.08ms	-	unsure 🔍 -1% - +6% -0.33ms - +1.65ms	unsure 🔍 -3% - +7% -0.74ms - +1.91ms
tip-of-tree tip-of-tree	27.03ms - 28.47ms	unsure 🔍 -6% - +1% -1.65ms - +0.33ms	-	unsure 🔍 -5% - +5% -1.43ms - +1.28ms
previous-release previous-release	26.68ms - 28.97ms	unsure 🔍 -7% - +3% -1.91ms - +0.74ms	unsure 🔍 -5% - +5% -1.28ms - +1.43ms	-

update

Version	Avg time	vs this-change	vs tip-of-tree tip-of-tree	vs previous-release previous-release
this-change	67.78ms - 72.19ms	-	unsure 🔍 -6% - +3% -4.38ms - +1.94ms	unsure 🔍 -6% - +3% -4.45ms - +1.92ms
tip-of-tree tip-of-tree	68.94ms - 73.47ms	unsure 🔍 -3% - +6% -1.94ms - +4.38ms	-	unsure 🔍 -5% - +4% -3.27ms - +3.18ms
previous-release previous-release	68.95ms - 73.54ms	unsure 🔍 -3% - +6% -1.92ms - +4.45ms	unsure 🔍 -4% - +5% -3.18ms - +3.27ms	-

nop-update

Version	Avg time	vs this-change	vs tip-of-tree tip-of-tree	vs previous-release previous-release
this-change	19.86ms - 21.80ms	-	unsure 🔍 -1% - +14% -0.08ms - +2.61ms	slower ❌ 2% - 18% 0.53ms - 3.33ms
tip-of-tree tip-of-tree	18.63ms - 20.49ms	unsure 🔍 -12% - +0% -2.61ms - +0.08ms	-	unsure 🔍 -4% - +11% -0.70ms - +2.04ms
previous-release previous-release	17.89ms - 19.90ms	faster ✔ 3% - 16% 0.53ms - 3.33ms	unsure 🔍 -10% - +3% -2.04ms - +0.70ms	-

this-change, tip-of-tree, previous-release

Browser: chrome-headless 115.0.5790.170
Sample size: 50
Built by: Benchmarks #5239
Commit: 4cdfecd

render

Version	Avg time	vs this-change	vs tip-of-tree tip-of-tree	vs previous-release previous-release
this-change	47.63ms - 48.55ms	-	faster ✔ 1% - 3% 0.33ms - 1.72ms	unsure 🔍 -3% - +0% -1.35ms - +0.21ms
tip-of-tree tip-of-tree	48.59ms - 49.63ms	slower ❌ 1% - 4% 0.33ms - 1.72ms	-	unsure 🔍 -1% - +3% -0.37ms - +1.27ms
previous-release previous-release	48.03ms - 49.29ms	unsure 🔍 -0% - +3% -0.21ms - +1.35ms	unsure 🔍 -3% - +1% -1.27ms - +0.37ms	-

update

Version	Avg time	vs this-change	vs tip-of-tree tip-of-tree	vs previous-release previous-release
this-change	99.69ms - 101.57ms	-	unsure 🔍 -2% - +0% -2.14ms - +0.50ms	unsure 🔍 -2% - +0% -2.29ms - +0.37ms
tip-of-tree tip-of-tree	100.53ms - 102.37ms	unsure 🔍 -0% - +2% -0.50ms - +2.14ms	-	unsure 🔍 -1% - +1% -1.45ms - +1.17ms
previous-release previous-release	100.66ms - 102.53ms	unsure 🔍 -0% - +2% -0.37ms - +2.29ms	unsure 🔍 -1% - +1% -1.17ms - +1.45ms	-

this-change, tip-of-tree, previous-release

Browser: chrome-headless 115.0.5790.170
Sample size: 50
Built by: Benchmarks #5239
Commit: 4cdfecd

render

Version	Avg time	vs this-change	vs tip-of-tree tip-of-tree	vs previous-release previous-release
this-change	44.38ms - 45.50ms	-	unsure 🔍 -1% - +2% -0.58ms - +0.95ms	unsure 🔍 -2% - +2% -0.88ms - +0.75ms
tip-of-tree tip-of-tree	44.23ms - 45.27ms	unsure 🔍 -2% - +1% -0.95ms - +0.58ms	-	unsure 🔍 -2% - +1% -1.04ms - +0.54ms
previous-release previous-release	44.41ms - 45.60ms	unsure 🔍 -2% - +2% -0.75ms - +0.88ms	unsure 🔍 -1% - +2% -0.54ms - +1.04ms	-

update

Version	Avg time	vs this-change	vs tip-of-tree tip-of-tree	vs previous-release previous-release
this-change	664.37ms - 670.80ms	-	unsure 🔍 -1% - +1% -4.24ms - +5.32ms	unsure 🔍 -1% - +1% -5.82ms - +5.26ms
tip-of-tree tip-of-tree	663.50ms - 670.59ms	unsure 🔍 -1% - +1% -5.32ms - +4.24ms	-	unsure 🔍 -1% - +1% -6.55ms - +4.91ms
previous-release previous-release	663.36ms - 672.38ms	unsure 🔍 -1% - +1% -5.26ms - +5.82ms	unsure 🔍 -1% - +1% -4.91ms - +6.55ms	-

update-reflect

Version	Avg time	vs this-change	vs tip-of-tree tip-of-tree	vs previous-release previous-release
this-change	672.37ms - 676.95ms	-	unsure 🔍 -1% - +0% -6.78ms - +0.33ms	unsure 🔍 -1% - +0% -7.58ms - +0.26ms
tip-of-tree tip-of-tree	675.17ms - 680.61ms	unsure 🔍 -0% - +1% -0.33ms - +6.78ms	-	unsure 🔍 -1% - +1% -4.62ms - +3.75ms
previous-release previous-release	675.14ms - 681.50ms	unsure 🔍 -0% - +1% -0.26ms - +7.58ms	unsure 🔍 -1% - +1% -3.75ms - +4.62ms	-

_{tachometer-reporter-action v2 for Benchmarks}

packages/labs/compiler/src/test/compile_lit_html_tests.ts

augustjk · 2023-08-16T18:52:15Z

packages/labs/compiler/src/test/compile_lit_html_tests.ts

Should this file actually be in packages/lit-html and we add @lit-labs/compiler to it as dev dependency? It is called from, and its output present in, packages/lit-html.

For compatibility with the 2.0 branch and 3.0 branch we cannot do this because lit-html uses TypeScript 4.x.
Thus any usage of the compiler from lit-html results in the transformer crashing because the TypeScript version is wrong.

I'd like to do what you suggest, but currently it's not possible except in the 3.0 branch. This is a temporary compromise to get runtime tests in.

Aha, right. I saw that note but totally glanced over the fact that the TS import would be project dependent. Thanks for clarifying!

packages/lit-html/package.json

packages/lit-html/src/test/directive-helpers_test.ts

augustjk

Excellent!

rictic

excellent!

packages/lit-html/package.json

lit-html tests are now compiled and tested and pass

b50c131

AndrewJakubowicz requested a review from kevinpschaaf as a code owner August 15, 2023 21:21

AndrewJakubowicz added 2 commits August 15, 2023 14:22

ensure compiled tests are part of lit-html test npm command

2aec796

add empty changeset

4bf8cbf

AndrewJakubowicz changed the title ~~[lit-labs/compiler] lit-html tests are now compiled and tested~~ [infra][lit-labs/compiler] lit-html tests are now also compiled and tested Aug 15, 2023

update some comments

909b113

AndrewJakubowicz requested review from augustjk, justinfagnani and rictic August 15, 2023 22:20

augustjk reviewed Aug 16, 2023

View reviewed changes

code review feedback

4cdfecd

augustjk approved these changes Aug 16, 2023

View reviewed changes

rictic approved these changes Aug 17, 2023

View reviewed changes

packages/lit-html/package.json Outdated Show resolved Hide resolved

code review feedback

a888ebf

AndrewJakubowicz enabled auto-merge (squash) August 17, 2023 21:58

AndrewJakubowicz merged commit 5354ab1 into main Aug 17, 2023

AndrewJakubowicz deleted the test-lit-html-compiled branch August 17, 2023 22:13

[infra][lit-labs/compiler] lit-html tests are now also compiled and tested #4102

[infra][lit-labs/compiler] lit-html tests are now also compiled and tested #4102

Uh oh!

Conversation

AndrewJakubowicz commented Aug 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Context

Features

Test plan

Future consideration

Uh oh!

changeset-bot bot commented Aug 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

github-actions bot commented Aug 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📊 Tachometer Benchmark Results

Summary

nop-update

render

update

update-reflect

Results

render

update

update-reflect

render

update

nop-update

render

update

render

update

update-reflect

Uh oh!

Uh oh!

augustjk Aug 16, 2023

Choose a reason for hiding this comment

Uh oh!

AndrewJakubowicz Aug 16, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

augustjk Aug 16, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

augustjk left a comment

Choose a reason for hiding this comment

Uh oh!

rictic left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

AndrewJakubowicz commented Aug 15, 2023 •

edited

Loading

changeset-bot bot commented Aug 15, 2023 •

edited

Loading

github-actions bot commented Aug 15, 2023 •

edited

Loading

AndrewJakubowicz Aug 16, 2023 •

edited

Loading