[chore] Fix package/benchmarks by updating chromedriver and package-lock #2439

AndrewJakubowicz · 2022-01-21T19:30:56Z

Context

Our benchmarks CI has recently broken as the chrome version has exceeded the chrome driver.

I.e. see example error: https://github.com/lit/lit/runs/4899938143?check_suite_focus=true#step:12:41

Error pasted: SessionNotCreatedError: session not created: This version of ChromeDriver only supports Chrome version 95

Testing

Reproduced the error locally, and fixed by updating package.json and package-lock.json.
Expect this PR to fix the Benchmarks/benchmarks github action.

Risk

Internal dev dependency update for benchmarking. Risk should be low.

Changelog

This is an internal tooling update - I don't think this needs a changelog.

changeset-bot · 2022-01-21T19:30:57Z

⚠️ No Changeset found

Latest commit: 401c99b

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

github-actions · 2022-01-21T19:31:18Z

📊 Tachometer Benchmark Results

Summary

nop-update

lit-html-kitchen-sink: unsure 🔍 -12% - +10% (-4.34ms - +3.66ms)
^{this-change vs tip-of-tree}

render

lit-element-list: unsure 🔍 -2% - +2% (-1.97ms - +1.86ms)
^{this-change vs tip-of-tree}
lit-html-kitchen-sink: unsure 🔍 -7% - +5% (-3.17ms - +2.31ms)
^{this-change vs tip-of-tree}
lit-html-repeat: unsure 🔍 -5% - +3% (-0.74ms - +0.42ms)
^{this-change vs tip-of-tree}
lit-html-template-heavy: unsure 🔍 -1% - +3% (-0.65ms - +1.91ms)
^{this-change vs tip-of-tree}
reactive-element-list: unsure 🔍 -1% - +3% (-0.85ms - +2.38ms)
^{this-change vs tip-of-tree}

update

lit-element-list: unsure 🔍 -2% - +1% (-17.64ms - +10.73ms)
^{this-change vs tip-of-tree}
lit-html-kitchen-sink: unsure 🔍 -6% - +7% (-7.79ms - +8.62ms)
^{this-change vs tip-of-tree}
lit-html-repeat: unsure 🔍 -1% - +1% (-5.18ms - +3.95ms)
^{this-change vs tip-of-tree}
lit-html-template-heavy: unsure 🔍 -2% - +2% (-3.82ms - +3.47ms)
^{this-change vs tip-of-tree}
reactive-element-list: unsure 🔍 -1% - +2% (-7.92ms - +24.06ms)
^{this-change vs tip-of-tree}

update-reflect

lit-element-list: unsure 🔍 -1% - +1% (-16.74ms - +6.56ms)
^{this-change vs tip-of-tree}
reactive-element-list: unsure 🔍 -1% - +2% (-15.43ms - +18.58ms)
^{this-change vs tip-of-tree}

Results

lit-element-list

Browser: chrome-headless 97.0.4692.71
Sample size: 50
Built by: Benchmarks #2242
Commit: 63d5e67

render

Version	Avg time	vs this-change	vs tip-of-tree tip-of-tree	vs previous-release previous-release
this-change	106.29ms - 109.38ms	-	unsure 🔍 -2% - +2% -1.97ms - +1.86ms	faster ✔ 20% - 23% 27.59ms - 31.67ms
tip-of-tree tip-of-tree	106.76ms - 109.02ms	unsure 🔍 -2% - +2% -1.86ms - +1.97ms	-	faster ✔ 20% - 23% 27.83ms - 31.32ms
previous-release previous-release	136.13ms - 138.79ms	slower ❌ 25% - 30% 27.59ms - 31.67ms	slower ❌ 26% - 29% 27.83ms - 31.32ms	-

update

Version	Avg time	vs this-change	vs tip-of-tree tip-of-tree	vs previous-release previous-release
this-change	1032.97ms - 1053.86ms	-	unsure 🔍 -2% - +1% -17.64ms - +10.73ms	faster ✔ 6% - 9% 72.07ms - 99.37ms
tip-of-tree tip-of-tree	1037.28ms - 1056.47ms	unsure 🔍 -1% - +2% -10.73ms - +17.64ms	-	faster ✔ 6% - 8% 69.26ms - 95.27ms
previous-release previous-release	1120.36ms - 1137.92ms	slower ❌ 7% - 10% 72.07ms - 99.37ms	slower ❌ 7% - 9% 69.26ms - 95.27ms	-

update-reflect

Version	Avg time	vs this-change	vs tip-of-tree tip-of-tree	vs previous-release previous-release
this-change	1129.14ms - 1145.40ms	-	unsure 🔍 -1% - +1% -16.74ms - +6.56ms	faster ✔ 4% - 6% 50.06ms - 73.83ms
tip-of-tree tip-of-tree	1134.01ms - 1150.70ms	unsure 🔍 -1% - +1% -6.56ms - +16.74ms	-	faster ✔ 4% - 6% 44.82ms - 68.89ms
previous-release previous-release	1190.54ms - 1207.88ms	slower ❌ 4% - 7% 50.06ms - 73.83ms	slower ❌ 4% - 6% 44.82ms - 68.89ms	-

lit-html-kitchen-sink

Browser: chrome-headless 97.0.4692.71
Sample size: 30
Built by: Benchmarks #2242
Commit: 63d5e67

render

Version	Avg time	vs this-change	vs tip-of-tree tip-of-tree	vs previous-release previous-release
this-change	44.72ms - 47.90ms	-	unsure 🔍 -7% - +5% -3.17ms - +2.31ms	faster ✔ 6% - 14% 3.08ms - 7.28ms
tip-of-tree tip-of-tree	44.51ms - 48.96ms	unsure 🔍 -5% - +7% -2.31ms - +3.17ms	-	faster ✔ 4% - 14% 2.14ms - 7.36ms
previous-release previous-release	50.12ms - 52.85ms	slower ❌ 6% - 16% 3.08ms - 7.28ms	slower ❌ 4% - 16% 2.14ms - 7.36ms	-

update

Version	Avg time	vs this-change	vs tip-of-tree tip-of-tree	vs previous-release previous-release
this-change	115.63ms - 126.77ms	-	unsure 🔍 -6% - +7% -7.79ms - +8.62ms	unsure 🔍 -8% - +5% -9.29ms - +6.17ms
tip-of-tree tip-of-tree	114.75ms - 126.81ms	unsure 🔍 -7% - +6% -8.62ms - +7.79ms	-	unsure 🔍 -8% - +5% -10.05ms - +6.09ms
previous-release previous-release	117.39ms - 128.12ms	unsure 🔍 -5% - +8% -6.17ms - +9.29ms	unsure 🔍 -5% - +8% -6.09ms - +10.05ms	-

nop-update

Version	Avg time	vs this-change	vs tip-of-tree tip-of-tree	vs previous-release previous-release
this-change	33.86ms - 38.96ms	-	unsure 🔍 -12% - +10% -4.34ms - +3.66ms	unsure 🔍 -12% - +2% -4.68ms - +0.64ms
tip-of-tree tip-of-tree	33.66ms - 39.83ms	unsure 🔍 -10% - +12% -3.66ms - +4.34ms	-	unsure 🔍 -13% - +4% -4.86ms - +1.50ms
previous-release previous-release	37.67ms - 39.18ms	unsure 🔍 -2% - +13% -0.64ms - +4.68ms	unsure 🔍 -4% - +14% -1.50ms - +4.86ms	-

lit-html-repeat

Browser: chrome-headless 97.0.4692.71
Sample size: 30
Built by: Benchmarks #2242
Commit: 63d5e67

render

Version	Avg time	vs this-change	vs tip-of-tree tip-of-tree	vs previous-release previous-release
this-change	13.38ms - 14.12ms	-	unsure 🔍 -5% - +3% -0.74ms - +0.42ms	faster ✔ 9% - 17% 1.41ms - 2.68ms
tip-of-tree tip-of-tree	13.46ms - 14.36ms	unsure 🔍 -3% - +5% -0.42ms - +0.74ms	-	faster ✔ 8% - 16% 1.21ms - 2.57ms
previous-release previous-release	15.28ms - 16.31ms	slower ❌ 10% - 20% 1.41ms - 2.68ms	slower ❌ 8% - 19% 1.21ms - 2.57ms	-

update

Version	Avg time	vs this-change	vs tip-of-tree tip-of-tree	vs previous-release previous-release
this-change	362.91ms - 368.71ms	-	unsure 🔍 -1% - +1% -5.18ms - +3.95ms	faster ✔ 31% - 32% 161.85ms - 174.29ms
tip-of-tree tip-of-tree	362.90ms - 369.94ms	unsure 🔍 -1% - +1% -3.95ms - +5.18ms	-	faster ✔ 30% - 32% 160.92ms - 173.99ms
previous-release previous-release	528.37ms - 539.38ms	slower ❌ 44% - 48% 161.85ms - 174.29ms	slower ❌ 44% - 48% 160.92ms - 173.99ms	-

lit-html-template-heavy

Browser: chrome-headless 97.0.4692.71
Sample size: 50
Built by: Benchmarks #2242
Commit: 63d5e67

render

Version	Avg time	vs this-change	vs tip-of-tree tip-of-tree	vs previous-release previous-release
this-change	71.30ms - 73.33ms	-	unsure 🔍 -1% - +3% -0.65ms - +1.91ms	faster ✔ 14% - 18% 12.17ms - 15.29ms
tip-of-tree tip-of-tree	70.90ms - 72.47ms	unsure 🔍 -3% - +1% -1.91ms - +0.65ms	-	faster ✔ 15% - 18% 12.94ms - 15.79ms
previous-release previous-release	84.86ms - 87.24ms	slower ❌ 17% - 21% 12.17ms - 15.29ms	slower ❌ 18% - 22% 12.94ms - 15.79ms	-

update

Version	Avg time	vs this-change	vs tip-of-tree tip-of-tree	vs previous-release previous-release
this-change	162.95ms - 167.77ms	-	unsure 🔍 -2% - +2% -3.82ms - +3.47ms	faster ✔ 11% - 14% 20.00ms - 27.13ms
tip-of-tree tip-of-tree	162.80ms - 168.28ms	unsure 🔍 -2% - +2% -3.47ms - +3.82ms	-	faster ✔ 10% - 14% 19.59ms - 27.18ms
previous-release previous-release	186.29ms - 191.55ms	slower ❌ 12% - 17% 20.00ms - 27.13ms	slower ❌ 12% - 17% 19.59ms - 27.18ms	-

reactive-element-list

Browser: chrome-headless 97.0.4692.71
Sample size: 50
Built by: Benchmarks #2242
Commit: 63d5e67

render

Version	Avg time	vs this-change	vs tip-of-tree tip-of-tree	vs previous-release previous-release
this-change	76.20ms - 78.50ms	-	unsure 🔍 -1% - +3% -0.85ms - +2.38ms	unsure 🔍 -0% - +4% -0.06ms - +2.77ms
tip-of-tree tip-of-tree	75.45ms - 77.71ms	unsure 🔍 -3% - +1% -2.38ms - +0.85ms	-	unsure 🔍 -1% - +3% -0.81ms - +1.99ms
previous-release previous-release	75.17ms - 76.82ms	unsure 🔍 -4% - +0% -2.77ms - +0.06ms	unsure 🔍 -3% - +1% -1.99ms - +0.81ms	-

update

Version	Avg time	vs this-change	vs tip-of-tree tip-of-tree	vs previous-release previous-release
this-change	1090.71ms - 1115.63ms	-	unsure 🔍 -1% - +2% -7.92ms - +24.06ms	unsure 🔍 -1% - +2% -11.63ms - +22.43ms
tip-of-tree tip-of-tree	1085.08ms - 1105.12ms	unsure 🔍 -2% - +1% -24.06ms - +7.92ms	-	unsure 🔍 -2% - +1% -18.01ms - +12.66ms
previous-release previous-release	1086.17ms - 1109.38ms	unsure 🔍 -2% - +1% -22.43ms - +11.63ms	unsure 🔍 -1% - +2% -12.66ms - +18.01ms	-

update-reflect

Version	Avg time	vs this-change	vs tip-of-tree tip-of-tree	vs previous-release previous-release
this-change	1206.46ms - 1228.82ms	-	unsure 🔍 -1% - +2% -15.43ms - +18.58ms	unsure 🔍 -2% - +1% -23.08ms - +10.22ms
tip-of-tree tip-of-tree	1203.26ms - 1228.88ms	unsure 🔍 -2% - +1% -18.58ms - +15.43ms	-	unsure 🔍 -2% - +1% -25.79ms - +9.78ms
previous-release previous-release	1211.74ms - 1236.41ms	unsure 🔍 -1% - +2% -10.22ms - +23.08ms	unsure 🔍 -1% - +2% -9.78ms - +25.79ms	-

_{tachometer-reporter-action v2 for Benchmarks}

Fix benchmarks CI by updating chromedriver and package-lock

401c99b

AndrewJakubowicz requested a review from justinfagnani as a code owner January 21, 2022 19:30

aomarks approved these changes Jan 21, 2022

View reviewed changes

AndrewJakubowicz merged commit a7ae45d into main Jan 22, 2022

AndrewJakubowicz deleted the update-chromedriver branch January 22, 2022 01:45

AndrewJakubowicz mentioned this pull request Apr 6, 2022

[chore] Fix package/benchmarks by updating chromedriver and package-lock #2705

Merged

[chore] Fix package/benchmarks by updating chromedriver and package-lock #2439

[chore] Fix package/benchmarks by updating chromedriver and package-lock #2439

Uh oh!

Conversation

AndrewJakubowicz commented Jan 21, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Context

Testing

Risk

Changelog

Uh oh!

changeset-bot bot commented Jan 21, 2022

⚠️ No Changeset found

Uh oh!

github-actions bot commented Jan 21, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📊 Tachometer Benchmark Results

Summary

nop-update

render

update

update-reflect

Results

render

update

update-reflect

render

update

nop-update

render

update

render

update

render

update

update-reflect

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

AndrewJakubowicz commented Jan 21, 2022 •

edited

Loading

github-actions bot commented Jan 21, 2022 •

edited

Loading