This issue is to track the product and engineering decision of: "how do we want long-term stable releases to work and be tested"?

(Any follow-up engineering work will be discussed elsewhere)

While fixing #167288, the release and infra folks [re]-discovered that, due to historic infrastructure decisions, test and build targets that do not exist on Tip-of-Tree (read: HEAD on the master channel) are not run, even on release candidate branches (for example, flutter-3.29-candidate.0, which is our current stable release).

What that means, concretely, is imagine (this is based on the real situation today):

# .ci.yaml for 3.29.0-candidate.0, which is "stable"
targets:
  - name: web_html_renderer_tests
  - name: web_canvas_kit_tests
  - name: web_sk_wasm_tests

Now, X months later, we remove support for html_renderer:

# .ci.yaml for master
targets:
  - name: web_canvas_kit_tests
  - name: web_sk_wasm_tests

The way we handle this today, is that (silently) web_html_renderer_tests is removed from our backend infrastructure (LUCI configuration), and Cocoon (silently) skips these builders, even in PRs and cherry-picks to flutter-3.29.0-candidate.0, because we have no way to run the builds.

So, if we wanted to know "do all the features released in 3.29-candidate work", we can no longer answer that question, at least not in automated fashion. In the case of the (already deprecated) HTML renderer, maybe we don't care too much, but I can imagine we (a) do care for other features and (b) sometimes this is just due to test re-sharding, meaning the tests still exist, they have just been renamed.

Potentially/maybe helpful; @jmagman pointed out (Googler only) go/flutter-build-test-infra did have a goal around this.