Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Gold service is flaky in post submit #108905

Closed
@Piinks

Description

@Piinks

Skia bug: https://bugs.chromium.org/p/skia/issues/detail?id=13621

We keep getting test failures for approved images on the master branch in post submit.
The reported failing image digests are actually positive image results when we go to the dashboard:

Here is one example of output:

01:26 +2277 ~14 -7: /opt/s/w/ir/x/w/flutter/packages/flutter/test/cupertino/text_field_test.dart: Overflow clipBehavior none golden                                                                    
══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following _Exception was thrown while running async test code:
Exception: Skia Gold received an unapproved image in post-submit
testing. Golden file images in flutter/flutter are triaged
in pre-submit during code review for the given PR.

Visit https://flutter-gold.skia.org/ to view and approve
the image(s), or revert the associated change. For more
information, visit the wiki:
https://github.com/flutter/flutter/wiki/Writing-a-golden-file-test-for-package:flutter

Debug information for Gold:
stdout: Given image with hash 00473a05465d2335a65840d025317772 for test
cupertino.overflow_clipbehavior_none.cupertino.0
Untriaged or negative image:
https://flutter-gold.skia.org/detail?test=cupertino.overflow_clipbehavior_none.cupertino.0&digest=00473a05465d2335a65840d025317772


stderr: Test: cupertino.overflow_clipbehavior_none.cupertino.0 FAIL

When the exception was thrown, this was the stack:
#0      SkiaGoldClient.imgtestAdd (package:flutter_goldens_client/skia_client.dart:218:7)
<asynchronous suspension>
<asynchronous suspension>
(elided one frame from package:stack_trace)
════════════════════════════════════════════════════════════════════════════════════════════════════

01:26 +2277 ~14 -8: /opt/s/w/ir/x/w/flutter/packages/flutter/test/cupertino/text_field_test.dart: Overflow clipBehavior none golden [E]                                                                
  Test failed. See exception logs above.
  The test description was: Overflow clipBehavior none golden

Following the link to the image test result shows an approved image though, so it is not clear why Gold says the test is failing.

Here are several builds where this has happened on multiple golden file tests:

https://ci.chromium.org/ui/p/flutter/builders/prod/Windows%20framework_tests_libraries/6123/overview
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20framework_tests_widgets/6225/overview
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_canvaskit_tests_3/3404/overview
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_canvaskit_tests_7_last/3382/overview

When re-run, the test is passing. This flaky behavior is breaking our tree.

CC @kjlubick & @LeandroLovisolo

Metadata

Metadata

Assignees

Labels

P0Critical issues such as a build break or regressiona: tests"flutter test", flutter_test, or one of our testsc: contributor-productivityTeam-specific productivity, code health, technical debt.c: flakeTests that sometimes, but not always, incorrectly passframeworkflutter/packages/flutter repository. See also f: labels.team-infraOwned by Infrastructure team

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions