Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Reland flaky golden file tests #115004

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 20 commits into from
Closed

Reland flaky golden file tests #115004

wants to merge 20 commits into from

Conversation

Piinks
Copy link
Contributor

@Piinks Piinks commented Nov 9, 2022 •
edited
Loading

Reland #114450
Reverted in #114902

WIP - Still trying to diagnose why https://flutter-gold.skia.org/detail?test=cupertino.date_picker_test.date.initial&digest=6cce8fe0712d4b12c137063cae36eccf closed the tree

Pre-launch Checklist

  • I read the Contributor Guide and followed the process outlined there for submitting PRs.
  • I read the Tree Hygiene wiki page, which explains my responsibilities.
  • I read and followed the Flutter Style Guide, including Features we expect every widget to implement.
  • I signed the CLA.
  • I listed at least one issue that this PR fixes in the description above.
  • I updated/added relevant documentation (doc comments with ///).
  • I added new tests to check the change I am making, or this PR is test-exempt.
  • All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

@flutter-dashboard flutter-dashboard bot added a: tests "flutter test", flutter_test, or one of our tests d: api docs Issues with https://api.flutter.dev/ d: examples Sample code and demos documentation f: cupertino flutter/packages/flutter/cupertino repository framework flutter/packages/flutter repository. See also f: labels. c: contributor-productivity Team-specific productivity, code health, technical debt. tool Affects the "flutter" command-line tool. See also t: labels. labels Nov 9, 2022
@Piinks
Copy link
Contributor Author

Piinks commented Nov 9, 2022 •
edited
Loading

@yjbanov I saw this was reverted after the tree closed on https://flutter-gold.skia.org/detail?test=cupertino.date_picker_test.date.initial&digest=6cce8fe0712d4b12c137063cae36eccf

According to our assumptions, the fuzzy settings should have passed the image.. I've reach out to @kjlubick to see if we're wrong here to think so.

@Piinks
Copy link
Contributor Author

Piinks commented Nov 9, 2022

Filed https://bugs.chromium.org/p/skia/issues/detail?id=13906&q=katelovett&can=2 for follow up on the Gold team. The fuzzy auto-triage should have occurred.

One thing we could do, @yjbanov WDYT of when flaky, removing the --passfail flag? That would prevent the tree from breaking, but it would not auto-triage the image in this case where fuzzy matching is not approving the image.

That means untriaged images could accrue on the dashboard, and left unattended, could alert other unrelated PRs... not sure how you would like to proceed 🤔

@Piinks Piinks mentioned this pull request Nov 17, 2022
Closed
@Piinks
Copy link
Contributor Author

Piinks commented Nov 17, 2022 •
edited
Loading

I heard back from the gold team, this is blocked on #93263.
The fuzzy matching won't auto-triage when the image sizes vary

@christopherfujino
Copy link
Contributor

I heard back from the gold team, this is blocked on #93263.

@Piinks should we leave this open optimistically or just leave a TODO?

@Piinks
Copy link
Contributor Author

Piinks commented Dec 2, 2022

I am going to close it. @yjbanov mentioned that #93263 may be the root cause of the flaky tests to begin with, so we can reassess after that is resolved.

@Piinks Piinks closed this Dec 2, 2022
@Piinks Piinks mentioned this pull request Dec 2, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
a: tests "flutter test", flutter_test, or one of our tests c: contributor-productivity Team-specific productivity, code health, technical debt. d: api docs Issues with https://api.flutter.dev/ d: examples Sample code and demos f: cupertino flutter/packages/flutter/cupertino repository framework flutter/packages/flutter repository. See also f: labels. tool Affects the "flutter" command-line tool. See also t: labels.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@Piinks @christopherfujino @atsansone @yjbanov