Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

neisw
Copy link
Contributor

@neisw neisw commented Sep 23, 2025

Reverts #2474

Per OpenShift policy, we are reverting this breaking change to get CI and/or nightly payloads flowing again.

Payload failures beginning with 4.21.0-0.nightly-2025-09-22-183616 with failures like:

Sep 22 21:24:08.217 E clusteroperator/insights condition/Available reason/UploadFailed status/False Unable to report: gateway server reported unexpected error code: 500 (request=): <html>
\n<head><title>500 Internal Server Error</title></head>
\n<body>
\n<center><h1>500 Internal Server Error</h1></center>
\n<hr><center>openresty</center>
\n</body>
\n</html>

To unrevert this, revert this PR, and layer an additional separate commit on top that addresses the problem. Before merging the unrevert, please run these jobs on the PR and check the result of these jobs to confirm the fix has corrected the problem:

/payload-job periodic-ci-openshift-release-master-nightly-4.21-e2e-aws-ovn-upgrade-fips

CC: @opokornyy

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Sep 23, 2025
@openshift-ci-robot
Copy link

openshift-ci-robot commented Sep 23, 2025

@neisw: This pull request references TRT-2313 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the bug to target the "4.21.0" version, but no target version was set.

In response to this:

Reverts #2474

Per OpenShift policy, we are reverting this breaking change to get CI and/or nightly payloads flowing again.

Payload failures beginning with 4.21.0-0.nightly-2025-09-22-183616 with failures like:

Sep 22 21:24:08.217 E clusteroperator/insights condition/Available reason/UploadFailed status/False Unable to report: gateway server reported unexpected error code: 500 (request=): <html>
\n<head><title>500 Internal Server Error</title></head>
\n<body>
\n<center><h1>500 Internal Server Error</h1></center>
\n<hr><center>openresty</center>
\n</body>
\n</html>

To unrevert this, revert this PR, and layer an additional separate commit on top that addresses the problem. Before merging the unrevert, please run these jobs on the PR and check the result of these jobs to confirm the fix has corrected the problem:

/payload-job periodic-ci-openshift-release-master-nightly-4.21-e2e-aws-ovn-upgrade-fips

CC: @opokornyy

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Copy link
Contributor

openshift-ci bot commented Sep 23, 2025

Hello @neisw! Some important instructions when contributing to openshift/api:
API design plays an important part in the user experience of OpenShift and as such API PRs are subject to a high level of scrutiny to ensure they follow our best practices. If you haven't already done so, please review the OpenShift API Conventions and ensure that your proposed changes are compliant. Following these conventions will help expedite the api review process for your PR.

@neisw
Copy link
Contributor Author

neisw commented Sep 23, 2025

/payload-job periodic-ci-openshift-release-master-nightly-4.21-e2e-aws-ovn-upgrade-fips

Copy link
Contributor

openshift-ci bot commented Sep 23, 2025

@neisw: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.21-e2e-aws-ovn-upgrade-fips

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/623e1830-9869-11f0-99c5-e0d3f75ef46a-0

@openshift-ci openshift-ci bot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Sep 23, 2025
Copy link
Contributor

openshift-ci bot commented Sep 23, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign joelspeed for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@JoelSpeed
Copy link
Contributor

@neisw Anything specifically pointing to this PR as being the culprit?

Our tooling for feature gates is designed such that if any cluster component attempts to read a feature gate that doesn't exist, it panics. The linked failures don't show the undiagnosed panic in pod I'd expect to see.

It's quite possible that some component isn't following the correct pattern, but I'd like to understand where that is if there is something holding FGs wrong

@neisw
Copy link
Contributor Author

neisw commented Sep 23, 2025

@JoelSpeed - suspicion at the moment is proximity to the operator failing and noting that the original pr did not pass a minor upgrade. I don't have absolute proof yet but wanted @opokornyy to have a look as well while running the payload job.

@neisw
Copy link
Contributor Author

neisw commented Sep 23, 2025

/payload-job periodic-ci-openshift-release-master-ci-4.21-upgrade-from-stable-4.20-e2e-gcp-ovn-rt-upgrade

Copy link
Contributor

openshift-ci bot commented Sep 23, 2025

@neisw: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-ci-4.21-upgrade-from-stable-4.20-e2e-gcp-ovn-rt-upgrade

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/b2fb0e90-9887-11f0-9c36-c55047959703-0

@neisw
Copy link
Contributor Author

neisw commented Sep 24, 2025

/hold
payload failures have not reoccurred and testing here is inconclusive

@openshift-ci openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Sep 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants