`Linux_samsung_a02 openpay_benchmarks__scroll_perf` keeps failing #108391

keyonghan · 2022-07-26T20:46:04Z

https://ci.chromium.org/p/flutter/builders/staging/Linux_samsung_a02%20openpay_benchmarks__scroll_perf

@iskakaushik could you help take a look? This has been causing testbed quarantine: https://chromium-swarm.appspot.com/bot?id=flutter-devicelab-linux-18 (/cc on call @godofredoc )

iskakaushik · 2022-07-26T20:51:42Z

Seems like the error is:

[openpay_benchmarks__scroll_perf] [STDOUT] stderr: [        ] FINE: Resolving dependencies finished (1.282s).
[openpay_benchmarks__scroll_perf] [STDOUT] stderr: [        ] ERR : Git error. Command: `git clone --mirror [email protected]:dnfield/vector_graphics.git /opt/s/w/ir/x/w/.pub-cache/git/cache/vector_graphics-48544a7914590401ff4792b7518970cfb4e75e40`
[openpay_benchmarks__scroll_perf] [STDOUT] stderr: [        ]     | stdout: 
[openpay_benchmarks__scroll_perf] [STDOUT] stderr: [        ]     | stderr: Cloning into bare repository '/opt/s/w/ir/x/w/.pub-cache/git/cache/vector_graphics-48544a7914590401ff4792b7518970cfb4e75e40'...
[openpay_benchmarks__scroll_perf] [STDOUT] stderr: [        ]     | Host key verification failed.
[openpay_benchmarks__scroll_perf] [STDOUT] stderr: [        ]     | fatal: Could not read from remote repository.
[openpay_benchmarks__scroll_perf] [STDOUT] stderr: [        ]     | 
[openpay_benchmarks__scroll_perf] [STDOUT] stderr: [        ]     | Please make sure you have the correct access rights
[openpay_benchmarks__scroll_perf] [STDOUT] stderr: [        ]     | and the repository exists.
[openpay_benchmarks__scroll_perf] [STDOUT] stderr: [        ]     | exit code: 128

iskakaushik · 2022-07-26T20:55:14Z

@keyonghan looks like pub is failing to resolve dependencies from github. This works for me locally. Are we allowed to clone from github.com in the LUCI bots?

@dnfield if we are not allowed to clone from github.com, is package:vector_graphics published somewhere? Maybe I can update the deps in openpay.

jonahwilliams · 2022-07-26T21:19:16Z

Its not published yet, but we could publish a new version to work around this issue

keyonghan · 2022-07-26T21:23:44Z

Our bots are not configured to clone from github.com by default:

git clone --mirror [email protected]:dnfield/vector_graphics.git vector_graphics-48544a7914590401ff4792b7518970cfb4e75e40
Cloning into bare repository 'vector_graphics-48544a7914590401ff4792b7518970cfb4e75e40'...
The authenticity of host 'github.com (192.30.255.112)' can't be established.
ED25519 key fingerprint is SHA256:+DiY3wvvV6TuJJhbpZisF/zLDA0zPMSvHdkr4UvCOqU.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])?

iskakaushik · 2022-07-26T21:40:25Z

@jonahwilliams could you please publish vector_graphics?

@keyonghan in that case I'll wait for the package to be published and update openpay to use that. Is there a way to reduce the amount of retries in the interim? Looks like its unbounded now.

jonahwilliams · 2022-07-26T21:58:24Z

I've published 0.0.1 for all vector graphics packages, you should be able to update the pubspec to reference these. There may be an additional git constraint for visibility_detector, but that can be removed as long as the package is updated to the latest version

iskakaushik · 2022-07-26T22:08:08Z

Updated OpenPay here: https://dash-internal-review.googlesource.com/c/openpay/+/1080, @keyonghan once this lands can we rerun one of the jobs and see if the issue gets resolved?

keyonghan · 2022-07-27T17:11:38Z

With the above CL, the test still fails: https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_samsung_a02%20openpay_benchmarks__scroll_perf/209/overview, but it is not clear to me what the error is as we are suppressing logs.

iskakaushik · 2022-07-27T17:17:00Z

@keyonghan is there a way to enable the logs for some users? to enable debugging this?

keyonghan · 2022-07-27T17:36:32Z

Here is one led run with logs: https://luci-milo.appspot.com/raw/build/logs.chromium.org/flutter/led/keyonghan_google.com/43a322025aa9e31c86541b291f1490ca2b3361cd40c872c22100a45d0547ed75/+/build.proto

     ] [   +9 ms] Skipping target: gen_dart_plugin_registrant
[openpay_benchmarks__scroll_perf] [STDOUT] stdout: [        ] [ +477 ms] kernel_snapshot: Starting due to {}
[openpay_benchmarks__scroll_perf] [STDOUT] stdout: [        ] [   +8 ms] /opt/s/w/ir/x/w/recipe_cleanup/tmpomvjfgv4/flutter sdk/bin/cache/dart-sdk/bin/dart --disable-dart-dev /opt/s/w/ir/x/w/recipe_cleanup/tmpomvjfgv4/flutter sdk/bin/cache/dart-sdk/bin/snapshots/frontend_server.dart.snapshot --sdk-root /opt/s/w/ir/x/w/recipe_cleanup/tmpomvjfgv4/flutter sdk/bin/cache/artifacts/engine/common/flutter_patched_sdk/ --target=flutter --no-print-incremental-dependencies -DFLUTTER_WEB_AUTO_DETECT=true -Ddart.vm.profile=true -Ddart.vm.product=false --compact-async --track-widget-creation --aot --tfa --packages /opt/s/w/ir/x/w/recipe_cleanup/tmp4s3ppzof/openpay/.dart_tool/package_config.json --output-dill /opt/s/w/ir/x/w/recipe_cleanup/tmp4s3ppzof/openpay/.dart_tool/flutter_build/d999e24a7fdecf060aac56889f67b2ff/app.dill --depfile /opt/s/w/ir/x/w/recipe_cleanup/tmp4s3ppzof/openpay/.dart_tool/flutter_build/d999e24a7fdecf060aac56889f67b2ff/kernel_snapshot.d --source file:///opt/s/w/ir/x/w/recipe_cleanup/tmp4s3ppzof/openpay/.dart_tool/flutter_build/dart_plugin_registrant.dart --source package:flutter/src/dart_plugin_registrant.dart -Dflutter.dart_plugin_registrant=file:///opt/s/w/ir/x/w/recipe_cleanup/tmp4s3ppzof/openpay/.dart_tool/flutter_build/dart_plugin_registrant.dart --verbosity=error file:///opt/s/w/ir/x/w/recipe_cleanup/tmp4s3ppzof/openpay/test_driver/scroll_perf.dart
[openpay_benchmarks__scroll_perf] [STDOUT] stderr: [+11797 ms] [+12659 ms] grpc/lib/src/client/http2_connection.dart:336:12: Error: The method 'isInsecureConnectionAllowed' isn't defined for the class '_SocketTransportConnector'.
[openpay_benchmarks__scroll_perf] [STDOUT] stderr: [        ] [        ]  - '_SocketTransportConnector' is from 'package:grpc/src/client/http2_connection.dart' ('grpc/lib/src/client/http2_connection.dart').
[openpay_benchmarks__scroll_perf] [STDOUT] stderr: [        ] [        ] Try correcting the name to the name of an existing method, or defining a method named 'isInsecureConnectionAllowed'.
[openpay_benchmarks__scroll_perf] [STDOUT] stderr: [        ] [        ]       if (!isInsecureConnectionAllowed(_host)) {
[openpay_benchmarks__scroll_perf] [STDOUT] stderr: [        ] [        ]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
[openpay_benchmarks__scroll_perf] [STDOUT] stdout: [+8999 ms] [+9038 ms] Persisting file store
[openpay_benchmarks__scroll_perf] [STDOUT] stdout: [ +100 ms] [  +19 ms] Done persisting file store
[openpay_benchmarks__scroll_perf] [STDOUT] stdout: [        ] [   +5 ms] "flutter assemble" took 22,417ms.
[openpay_benchmarks__scroll_perf] [STDOUT] stdout: [        ] [   +1 ms] ensureAnalyticsSent: 0ms
[openpay_benchmarks__scroll_perf] [STDOUT] stdout: [        ] [        ] Running shutdown hooks
[openpay_benchmarks__scroll_perf] [STDOUT] stdout: [        ] [        ] Shutdown hooks complete
[openpay_benchmarks__scroll_perf] [STDOUT] stdout: [        ] [        ] exiting with code 1
[openpay_benchmarks__scroll_perf] [STDOUT] stdout: [        ] Task :app:compileFlutterBuildProfile in app Finished
[openpay_benchmarks__scroll_perf] [STDOUT] stderr: [        ] [   +1 ms] Target kernel_snapshot failed: Exception
[openpay_benchmarks__scroll_perf] [STDOUT] stderr: [        ]            #0      KernelSnapshot.build (package:flutter_tools/src/build_system/targets/common.dart:245:7)
[openpay_benchmarks__scroll_perf] [STDOUT] stderr: [        ]            <asynchronous suspension>
[openpay_benchmarks__scroll_perf] [STDOUT] stderr: [        ]            #1      _BuildInstance._invokeInternal (package:flutter_tools/src/build_system/build_system.dart:839:9)
[openpay_benchmarks__scroll_perf] [STDOUT] stderr: [        ]            <asynchronous suspension>
[openpay_benchmarks__scroll_perf] [STDOUT] stderr: [        ]            #2      Future.wait.<anonymous closure> (dart:async/future.dart:522:21)
[openpay_benchmarks__scroll_perf] [STDOUT] stderr: [        ]            <asynchronous suspension>

iskakaushik · 2022-07-27T17:49:17Z

https://dash-internal-review.googlesource.com/c/openpay/+/1140 fixes this error.

iskakaushik · 2022-07-27T18:49:05Z

@keyonghan looks like it's still failing :(.

Is there a way for me to trigger runs using led and enable logs?
Can I use a custom revision of openpay for the test runs?
Could you please disable this test until I can get a successful run?

Thanks.

keyonghan · 2022-07-27T19:00:05Z

@keyonghan looks like it's still failing :(.

Is there a way for me to trigger runs using led and enable logs?

Commenting out https://cs.opensource.google/flutter/recipes/+/main:recipes/devicelab/devicelab_drone.py;l=78 will enable the logs.

Then you can run led based on the local changes.

Can I use a custom revision of openpay for the test runs?

You can try replacing ref: https://cs.opensource.google/flutter/recipes/+/main:recipes/devicelab/devicelab_drone.py;l=75, with the one from you attempting CL.

Could you please disable this test until I can get a successful run?

#108466

iskakaushik · 2022-08-01T13:56:01Z

I finally got around to trying it, I modified the properties of an existing builder and added the relevant bits to checkout openpay. I'm running into errors like so:

[E 2022-08-01 09:52:57] original error: googleapi: Error 403: user "[email protected]" does not have permission "swarming.tasks.createInRealm"

Maybe I need some permissions to launch tasks via CLI in prod. I tried changing prod to try and it still failed.

@keyonghan could you please help me with this?

keyonghan · 2022-08-01T17:18:19Z

Try is expected to work. How about staging? Could you share the command?

iskakaushik · 2022-08-01T17:26:19Z

I used cat properties.json | led edit-recipe-bundle | led launch, using https://paste.googleplex.com/5841871638102016 as the properties.json file. I changed line 26, from

          "bucket": "prod",

To

          "bucket": "try",

keyonghan · 2022-08-01T17:33:43Z

Line 26 is still using prod.

You can try:

led get-builder "luci.flutter.staging:Linux_android old_gallery__transition_perf" > /tmp/test.json
Update 1) task_name 2) device_type: SM-A025V
cat /tmp/test.json | led edit-recipe-bundle | led launch

iskakaushik · 2022-08-01T17:36:33Z

I did change line 26 to try before running it :), still failed.
Will attempt the solution you recommended.

update: it works now!

iskakaushik · 2022-08-01T21:28:46Z

https://dash-internal-review.googlesource.com/c/openpay/+/1160 should likely fix this issue, but I found another one while debugging this.

@keyonghan looks like suppress_logs means we don't initialize: initialize_logs_collection, which in-turn does not set FLUTTER_TEST_OUTPUTS_DIR, and the tests currently rely on this env var to be set to find the json files. Would it be OK to call initialize_logs_collection but just not upload the logs?

keyonghan · 2022-08-01T23:30:22Z

We should be good on that. Approved https://flutter-review.googlesource.com/c/recipes/+/32340.

iskakaushik · 2022-08-02T14:12:03Z

@keyonghan , this test seems to succeed now, but its still flaky on one of the bots:

https://paste.googleplex.com/5051984135585792

Any reason why the URL in the above mentioned logs wouldn't resolve?

keyonghan · 2022-08-02T16:21:14Z

There is no difference between linux18 and linux27. I would wait for a couple of runs to see if this is a network flake.

keyonghan · 2022-08-04T21:38:57Z

The test is failing consistently again on both linux18 and linux27: https://ci.chromium.org/p/flutter/builders/staging/Linux_samsung_a02%20openpay_benchmarks__scroll_perf

jmagman · 2022-09-09T20:27:56Z

This is still failing every commit, no logs though:
https://ci.chromium.org/p/flutter/builders/staging/Linux_samsung_a02%20openpay_benchmarks__scroll_perf/1800

keyonghan · 2022-09-12T17:24:54Z

@iskakaushik Shall we skip the test from CI before it is ready?

iskakaushik · 2022-09-12T17:36:32Z

@keyonghan yes, that sounds good to me

sealesj · 2022-09-12T20:04:33Z

@iskakaushik created PR to skip the test for now here #111415

keyonghan · 2023-04-20T18:15:20Z

@iskakaushik Do we still need this test?

godofredoc · 2023-08-22T01:54:47Z

Closing as obsolete.

github-actions · 2023-09-05T02:05:10Z

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

keyonghan assigned iskakaushik Jul 26, 2022

keyonghan added the customer: money (g3) label Jul 26, 2022

keyonghan mentioned this issue Jul 27, 2022

Remove Linux_samsung_a02 openpay_benchmarks__scroll_perf #108466

Merged

iskakaushik mentioned this issue Aug 1, 2022

Reland Linux_samsung_a02 openpay_benchmarks__scroll_perf (#108466) #108769

Merged

keyonghan mentioned this issue Aug 4, 2022

flutter-devicelab-linux-18 tests failed Task failed: DeviceException: No Android devices detected #109001

Closed

keyonghan changed the title ~~Linux_samsung_a02 openpay_benchmarks__scroll_perf keeps timing out~~ Linux_samsung_a02 openpay_benchmarks__scroll_perf keeps failing Aug 4, 2022

keyonghan mentioned this issue Sep 12, 2022

Remove Linux_samsung_a02 openpay_benchmarks__scroll_perf from CI #111404

Closed

danagbemava-nc added the team-infra Owned by Infrastructure team label Sep 27, 2022

keyonghan added the passed secondary triage label Apr 20, 2023

Hixie removed the passed secondary triage label Jul 8, 2023

godofredoc closed this as completed Aug 22, 2023

github-actions bot locked as resolved and limited conversation to collaborators Sep 5, 2023

Linux_samsung_a02 openpay_benchmarks__scroll_perf keeps failing #108391

Linux_samsung_a02 openpay_benchmarks__scroll_perf keeps failing #108391

Comments

keyonghan commented Jul 26, 2022

iskakaushik commented Jul 26, 2022

Uh oh!

iskakaushik commented Jul 26, 2022

Uh oh!

jonahwilliams commented Jul 26, 2022

Uh oh!

keyonghan commented Jul 26, 2022

Uh oh!

iskakaushik commented Jul 26, 2022

Uh oh!

jonahwilliams commented Jul 26, 2022

Uh oh!

iskakaushik commented Jul 26, 2022

Uh oh!

keyonghan commented Jul 27, 2022

Uh oh!

iskakaushik commented Jul 27, 2022

Uh oh!

keyonghan commented Jul 27, 2022

Uh oh!

iskakaushik commented Jul 27, 2022

Uh oh!

iskakaushik commented Jul 27, 2022

Uh oh!

keyonghan commented Jul 27, 2022

Uh oh!

iskakaushik commented Aug 1, 2022

Uh oh!

keyonghan commented Aug 1, 2022

Uh oh!

iskakaushik commented Aug 1, 2022

Uh oh!

keyonghan commented Aug 1, 2022

Uh oh!

iskakaushik commented Aug 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

iskakaushik commented Aug 1, 2022

Uh oh!

keyonghan commented Aug 1, 2022

Uh oh!

iskakaushik commented Aug 2, 2022

Uh oh!

keyonghan commented Aug 2, 2022

Uh oh!

keyonghan commented Aug 4, 2022

Uh oh!

jmagman commented Sep 9, 2022

Uh oh!

keyonghan commented Sep 12, 2022

Uh oh!

iskakaushik commented Sep 12, 2022

Uh oh!

sealesj commented Sep 12, 2022

Uh oh!

keyonghan commented Apr 20, 2023

Uh oh!

godofredoc commented Aug 22, 2023

Uh oh!

github-actions bot commented Sep 5, 2023

Uh oh!

`Linux_samsung_a02 openpay_benchmarks__scroll_perf` keeps failing #108391

`Linux_samsung_a02 openpay_benchmarks__scroll_perf` keeps failing #108391

iskakaushik commented Aug 1, 2022 •

edited

Loading