Skip to content

test: next.js repo tests adjustments #3168

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Oct 8, 2025
+57 −4
Merged

test: next.js repo tests adjustments #3168

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
6044c8c
test: pin @types/react(-dom) for specific vercel/next.js test when ru…
pieh Oct 2, 2025
026a5c6
test: adjust next.js repo test setup to follow deps installation more…
pieh Oct 2, 2025
7924f03
test: don't ignore buildArgs
pieh Oct 2, 2025
ebe626f
test: set IS_WEBPACK_TEST and IS_TURBOPACK_TEST
pieh Oct 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 22 additions & 1 deletion .github/workflows/test-e2e.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,24 @@ jobs:
echo "version=$NODE_VERSION" >> $GITHUB_OUTPUT
echo "Node version for 'next@${{ matrix.version_spec.selector }}' is '$NODE_VERSION'"

- name: Decide default bundler
id: decide-default-bundler
shell: bash
run: |
# we run tests with default bundler for given Next.js version
# some tests have webpack/turbopack specific assertions or skips
# so we need to set IS_WEBPACK_TEST, IS_TURBOPACK_TEST env vars accordingly
# to ensure tests assert behavior of the default bundler
DEFAULT_BUNDLER="webpack"
if [ "${{ matrix.version_spec.selector }}" = "canary" ]; then
# this is not ideal, because we set turbopack default just when explicitly using canary tag as target
# but next@canary are still on 15 major, so we can't yet use major version of resolved next version
# as condition
DEFAULT_BUNDLER="turbopack"
fi
echo "default_bundler=$DEFAULT_BUNDLER" >> $GITHUB_OUTPUT
echo "Default bundler for 'next@${{ matrix.version_spec.selector }}' is '$DEFAULT_BUNDLER'"

- name: setup node
uses: actions/setup-node@v5
with:
Expand Down Expand Up @@ -218,6 +236,9 @@ jobs:
# one job may wait for deploys in other jobs (only one deploy may be in progress for
# a given alias at a time), resulting in cascading timeouts.
DEPLOY_ALIAS: vercel-next-e2e-${{ matrix.version_spec.selector }}-${{ matrix.group }}
NEXT_RESOLVED_VERSION: ${{ matrix.version_spec.version }}
IS_WEBPACK_TEST: ${{ steps.decide-default-bundler.outputs.default_bundler == 'webpack' && '1' || '' }}
IS_TURBOPACK_TEST: ${{ steps.decide-default-bundler.outputs.default_bundler == 'turbopack' && '1' || '' }}
Comment on lines +240 to +241
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

those env vars are used by next test helpers and will impact isTurbopack etc checks - for example like one used in this test https://github.com/vercel/next.js/blob/a7d3aecafedc920221a885b8ab00f35f6de5e91c/test/e2e/esm-externals/esm-externals.test.ts

run: node run-tests.js -g ${{ matrix.group }}/${{ needs.setup.outputs.total }} -c ${TEST_CONCURRENCY} --type e2e
working-directory: ${{ env.next-path }}

Expand All @@ -226,7 +247,7 @@ jobs:
uses: actions/upload-artifact@v4
with:
name: test-result-${{matrix.version_spec.selector}}-${{ matrix.group }}
path: ${{ env.next-path }}/test/test-junit-report/*.xml
path: ${{ env.next-path }}/test/${{steps.decide-default-bundler.outputs.default_bundler == 'turbopack' && 'turbopack-test-junit-report' || 'test-junit-report'}}/*.xml
publish-test-results:
name: 'E2E Test Summary (${{matrix.version_spec.selector}})'
needs:
Expand Down
38 changes: 35 additions & 3 deletions tests/netlify-deploy.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import fs from 'fs-extra'
import { Span } from 'next/src/trace'
import { tmpdir } from 'node:os'
import path from 'path'
import { satisfies as satisfiesVersionRange } from 'semver'
import { NextInstance } from './base'

async function packNextRuntimeImpl() {
Expand Down Expand Up @@ -58,6 +59,34 @@ export class NextDeployInstance extends NextInstance {

const setupStartTime = Date.now()

const { runtimePackageName, runtimePackageTarballPath } = await packNextRuntime()

this.dependencies = {
...(this.dependencies || {}),
// add the runtime package as a dependency
[runtimePackageName]: `file:${runtimePackageTarballPath}`,
}
Comment on lines +64 to +68
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this isn't strictly needed change, but I do find it nicer to let next.js repo e2e utils to just produce package.json that has our runtime as dependency, instead of adding it as part of installation

especially as I used this.dependencies below to "fix" a test setup for us, so all the things messing with deps is using single way of doing that


if (
typeof this.files === 'string' &&
this.files.includes('back-forward-cache') &&
process.env.NEXT_RESOLVED_VERSION &&
satisfiesVersionRange(process.env.NEXT_RESOLVED_VERSION, '<=15.5.4')
) {
require('console').log('Pinning @types/react(-dom) for back-forward-cache test fixture')
// back-forward-cache test fixture is failing types checking because:
// - @types/react(-dom) types are not pinned
// - fixture uses react `unstable_Activity` export which since fixture was introduced is no longer unstable
// and types were updated for that and no longer provide types for that export (instead provide for `Activity`)
// this adds the pinning of types to version of types that still had `unstable_Activity` type
this.dependencies['@types/react'] = '19.1.1'
this.dependencies['@types/react-dom'] = '19.1.2'
}
Comment on lines +70 to +84
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is something that would fail on next.js repo as well if they run the test against current latest. The test was updated in canary, but not in latest.

I don't love this hack, but this test generally passes and I'd rather not skip it. Creating triaging issue doesn't make much sense either, so only option left is to "patch" the test when we run against current latest


if (!this.buildCommand && this.buildArgs && this.buildArgs.length > 0) {
this.buildCommand = `next build ${this.buildArgs.join(' ')}`
}
Comment on lines +86 to +88
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is to support this test https://github.com/vercel/next.js/blob/a7d3aecafedc920221a885b8ab00f35f6de5e91c/test/e2e/react-compiler/react-compiler.test.ts#L42 , without something like this --profile would not be applied and test rely on react profiling being enabled


// create the test site
await super.createTestDir({ parentSpan, skipInstall: true })

Expand All @@ -70,10 +99,13 @@ export class NextDeployInstance extends NextInstance {
await fs.rename(nodeModules, nodeModulesBak)
}

const { runtimePackageName, runtimePackageTarballPath } = await packNextRuntime()

// install dependencies
await execa('npm', ['i', runtimePackageTarballPath, '--legacy-peer-deps'], {
// --force is used to match behavior of `pnpm install --strict-peer-dependencies=false` used by Vercel
// there is a test fixture that have `@babel/preset-flow` as a dependency which has a peer dependency of `@babel/core@^7.0.0-0`,
// but `@babel/core` is not specified as dependency, so we need to automatically attempt to install peer dependencies
// but also not fail in case of peer dependency versions not matching for other fixtures, so `--legacy-peer-deps` is not good option here
// https://github.com/vercel/next.js/blob/7453d200579512a6574f9c53edd716e5cc01615c/test/e2e/babel/index.test.js#L7-L9
await execa('npm', ['install', '--force'], {
cwd: this.testDir,
stdio: 'inherit',
})
Expand Down
Loading