test: fix test setup #2749
Merged
test: fix test setup #2749
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
📊 Package size report No changes
Unchanged files
🤖 This report was automatically generated by pkg-size-action |
984d977 to
9e2e4e8
Compare
…r build a fixture
…repare script changes
9e2e4e8 to
0d9c1f4
Compare
…o test prepare script changes" This reverts commit 0d9c1f4.
e6e8614 to
f7892a9
Compare
bd9da1f to
1230176
Compare
1230176 to
948fd2b
Compare
mrstork
reviewed
Feb 12, 2025
| - name: setup pnpm/yarn | ||
| run: corepack enable | ||
| run: | | ||
| npm install -g corepack --force |
There was a problem hiding this comment.
Learning - why corepack@latest above, and then --force in this step?
There was a problem hiding this comment.
The @latest selector in the e2e is probably not needed - just result of iterating and trying to find working solution. I will remove it to keep things in sync.
The --force here is needed, because Windows one otherwise would fail to install - without it I was getting following error:
npm error code EEXIST
npm error path C:\npm\prefix\yarn
npm error EEXIST: file already exists
npm error File exists: C:\npm\prefix\yarn
npm error Remove the existing file and try again, or run npm
npm error with --force to overwrite files recklessly.
And we only needed it in jobs that run integration tests because only those use Windows runners. E2E ones are linux only hence don't need it.
mrstork
reviewed
Feb 12, 2025
| 'cli-before-regional-blobs-support', | ||
| 'dist-dir', | ||
| // There is also a bug on Windows on Node.js 18.20.6, that cause build failures on this fixture | ||
| // see https://github.com/opennextjs/opennextjs-netlify/actions/runs/13268839161/job/37043172448?pr=2749#step:12:78 |
There was a problem hiding this comment.
Is there a way to make this comment resolvable? For example, saying something more along the lines of "when so and so is merged in, we can remove this test from this set." Perhaps a link to a github or linear issue?
There was a problem hiding this comment.
There is currently no Next.js issue to link to as of now and I didn't find time to create proper Next.js issue for it myself (and I would need to prep repro and make sure it works on my personal windows machine, so that's not something I really have time to do during work hours :S). Without that issue in Next.js there is not much of a point creating internal linear issue, because it wouldn't say anything more than this comment is saying now.
Being a bit sneaky I could just remove this whole comment as it's still true that we don't actually use this fixture in integration tests and let someone possibly encounter that problem without warning, so adding this comment was just trying to balance providing at least some information here while acknowledging that I don't have bandwidth to create proper tracking ticket in Next.js repository.
mrstork
reviewed
Feb 12, 2025
tests/prepare.mjs
Outdated
| 'middleware-single-matcher', | ||
| 'nx-integrated', | ||
| 'turborepo', | ||
| // We do have a skipped test using this fixture (due to integration test setup not working for monorepos) |
There was a problem hiding this comment.
Same here, perhaps a linear issue link?
There was a problem hiding this comment.
Would removing the skipped test in this PR (the test was skipped a year ago - 219588e#diff-924485d81a4d8552389a0b2697f69ee3508807e55c08077e48455ea7e44be5d2R39 work?
As comment around reason for skipping mention - we actually do have e2e tests using this fixture proving the fixture is working
The test was skipped over a year ago 219588e#diff-924485d81a4d8552389a0b2697f69ee3508807e55c08077e48455ea7e44be5d2R36-R39 so this doesn't seem like actual priority that we would work on anytime soon.
There was a problem hiding this comment.
Ah I see, I understand the situation better now. I think the comment is actually fine as is, or we can remove the skipped test like you said, that would also work.
… have e2e that assert it is working
mrstork
approved these changes
Feb 12, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Our tests are currently failing - see last scheduled test run - https://github.com/opennextjs/opennextjs-netlify/actions/runs/13279231786
This PR contains couple of adjustments:
Upgrade corepack ( currently we are hitting Newly published versions of package managers distributed from npm cannot be installed due to key id mismatch nodejs/corepack#612 because corepack version shipped with used Node.js version doesn't contain the fix - only Node@23 ships with corepack version that has fix for it)
Skip building fixtures that are not used in our integration tests in our
preparescript - this is both making our CI runs a bit faster, but also it skipsmiddleware-ogfixture in paricular that currently fails to build on Windows on Node 18.20.6 - this is not next runtime issue - this is pure Next.js issue that is result of some Node.js changes that were done as security fix that unfortunately also broke Next's OpenGraph handling on Windows on that Node version. The same issue was kind of tackled in some capacity before in test: use branch/alias deploys for e2e test suite to enable automatic deploy cleanup #2752 (comment) that has a bit of context on the problem - build time issue (for different fixture) was not visible then because cached fixtures were used (that were created on previous Node.js patch version before the change that broke OpenGraph handling in Next)If prepare script fails to build a fixture - it now exits with non-zero code. We were not checking if our prepare script actually build test fixtures and as a result we not only attempt to run tests on broken fixtures - see https://github.com/opennextjs/opennextjs-netlify/actions/runs/12944298814/job/36105004178#step:11:291 failure that later result in test failures in same run, but also possibly caching that (if one of test shards don't fail any tests - which can happen for a shard that ... runs only unit tests - like this one https://github.com/opennextjs/opennextjs-netlify/actions/runs/12944298814/job/36105016872 )
There is still some root problem to solve on why npm installation fails on windows while it doesn't on linux (maybe some npm cache related problem?), so this is just attempting to short circuit CI when any of fixtures fail to build - avoid running tests and more importantly - avoid caching them (for unit test shards case), as subsequent test runs will have confusing errors about not exising "publish dir" and test run not even building fixtures and just pulling from cache
Documentation
Tests
For point 3:
I did intentionally break 2 fixtures with second commit:
next build(which we do run aspostinstallscript in our fixtures)Results can be seen in https://github.com/opennextjs/opennextjs-netlify/actions/runs/12996052794 with all integration and unit tests shards failing, not even proceeding to running tests and just skipping fixture caching
I later reverted intentional breakage