Skip to content

test: deflake and another next 16 compat update for fixtures #3181

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 5 commits into from
Closed

test: deflake and another next 16 compat update for fixtures #3181

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
ae33942
test: update to work with next@canary
pieh Oct 16, 2025
152decc
test: nx linux issue workaround
pieh Oct 16, 2025
76bccca
test: fetch-retry for github actions
pieh Oct 16, 2025
0e951a2
test: adjust Netlify Edge stale cache-status assertion
pieh Oct 16, 2025
e47ff48
fix: recursively wait for background work
pieh Oct 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/run-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,7 @@ jobs:
NETLIFY_AUTH_TOKEN: ${{ secrets.NETLIFY_AUTH_TOKEN }}
NEXT_VERSION: ${{ matrix.version }}
NEXT_RESOLVED_VERSION: ${{ steps.resolve-next-version.outputs.version }}
NODE_OPTIONS: --import ${{ github.workspace }}/tools/fetch-retry.mjs
- name: Upload blob report to GitHub Actions Artifacts
uses: actions/upload-artifact@v4
if: always()
Expand Down Expand Up @@ -220,13 +221,15 @@ jobs:
env:
NEXT_VERSION: ${{ matrix.version }}
NEXT_RESOLVED_VERSION: ${{ steps.resolve-next-version.outputs.version }}
NODE_OPTIONS: --import ${{ github.workspace }}/tools/fetch-retry.mjs
- name: 'Unit and integration tests'
run: npm run test:ci:unit-and-integration -- --shard=${{ matrix.shard }}/8
env:
NETLIFY_AUTH_TOKEN: ${{ secrets.NETLIFY_AUTH_TOKEN }}
NEXT_VERSION: ${{ matrix.version }}
NEXT_RESOLVED_VERSION: ${{ steps.resolve-next-version.outputs.version }}
TEMP: ${{ github.workspace }}/..
NODE_OPTIONS: --import ${{ github.workspace }}/tools/fetch-retry.mjs

smoke:
if: always()
Expand Down Expand Up @@ -289,6 +292,7 @@ jobs:
NETLIFY_AUTH_TOKEN: ${{ secrets.NETLIFY_AUTH_TOKEN }}
NEXT_VERSION: ${{ matrix.version }}
NEXT_RESOLVED_VERSION: ${{ steps.resolve-next-version.outputs.version }}
NODE_OPTIONS: --import ${{ github.workspace }}/tools/fetch-retry.mjs

merge-reports:
if: always()
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/test-e2e.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -239,6 +239,7 @@ jobs:
NEXT_RESOLVED_VERSION: ${{ matrix.version_spec.version }}
IS_WEBPACK_TEST: ${{ steps.decide-default-bundler.outputs.default_bundler == 'webpack' && '1' || '' }}
IS_TURBOPACK_TEST: ${{ steps.decide-default-bundler.outputs.default_bundler == 'turbopack' && '1' || '' }}
NODE_OPTIONS: --import ${{ github.workspace }}/${{ env.runtime-path}}/tools/fetch-retry.mjs
run: node run-tests.js -g ${{ matrix.group }}/${{ needs.setup.outputs.total }} -c ${TEST_CONCURRENCY} --type e2e
working-directory: ${{ env.next-path }}

Expand Down
22 changes: 16 additions & 6 deletions src/run/handlers/request-context.cts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,18 +67,28 @@ export function createRequestContext(request?: Request, context?: Context): Requ
logger.debug('[NetlifyNextRuntime] Background revalidation request')
}

async function recursivelyWaitForBackgroundWork() {
let settledPromisesCount = 0
while (settledPromisesCount < backgroundWorkPromises.length) {
const currentPromiseCount = backgroundWorkPromises.length
await Promise.allSettled(backgroundWorkPromises)
settledPromisesCount = currentPromiseCount
}
}

return {
isBackgroundRevalidation,
captureServerTiming: request?.headers.has('x-next-debug-logging') ?? false,
trackBackgroundWork: (promise) => {
if (context?.waitUntil) {
context.waitUntil(promise)
} else {
backgroundWorkPromises.push(promise)
}
backgroundWorkPromises.push(promise)
},
get backgroundWorkPromise() {
return Promise.allSettled(backgroundWorkPromises)
if (context?.waitUntil) {
// when context.waitUntil is available, we offload background work awaiting to it
context.waitUntil(recursivelyWaitForBackgroundWork())
return Promise.resolve()
}
return recursivelyWaitForBackgroundWork()
},
logger,
requestID: request?.headers.get('x-nf-request-id') ?? getFallbackRequestID(),
Expand Down
2 changes: 1 addition & 1 deletion tests/e2e/page-router.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -425,7 +425,7 @@ test.describe('Simple Page Router (no basePath, no i18n)', () => {
const response2 = await page.goto(new URL(pathname, pageRouter.url).href)
expect(response2?.status()).toBe(200)
expect(response2?.headers()['cache-status']).toMatch(
/("Netlify Edge"; hit; fwd=stale|"Netlify Durable"; hit; ttl=-[0-9]+)/m,
/("Netlify Edge"; fwd=stale|"Netlify Durable"; hit; ttl=-[0-9]+)/m,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the Netlify Edge one was wrong for quite some time so this in practice was relying on hitting durable cache case

)
expect(response2?.headers()['debug-netlify-cdn-cache-control']).toMatch(
/s-maxage=60, stale-while-revalidate=[0-9]+, durable/,
Expand Down
2 changes: 1 addition & 1 deletion tests/fixtures/use-cache/app/default/use-cache-component/dynamic/ttl-1year/[slug]/page.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { unstable_cacheLife as cacheLife, unstable_cacheTag as cacheTag } from 'next/cache'
import { cacheLife, cacheTag } from '../../../../../next-cache'
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unstable_cacheLife no longer exist in next@canary - it's just cacheLife now

But because we still will be running tests for next@15 I did add helper module for compatibility that will use either export with unstable_ prefix or stable one, depending on which is available in tested next.js version

import {
BasePageComponentProps,
getDataImplementation,
Expand Down
2 changes: 1 addition & 1 deletion ...s/fixtures/use-cache/app/default/use-cache-component/dynamic/ttl-5seconds/[slug]/page.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { unstable_cacheLife as cacheLife, unstable_cacheTag as cacheTag } from 'next/cache'
import { cacheLife, cacheTag } from '../../../../../next-cache'
import {
BasePageComponentProps,
getDataImplementation,
Expand Down
2 changes: 1 addition & 1 deletion ...s/fixtures/use-cache/app/default/use-cache-component/static/ttl-10seconds/[slug]/page.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { unstable_cacheLife as cacheLife, unstable_cacheTag as cacheTag } from 'next/cache'
import { cacheLife, cacheTag } from '../../../../../next-cache'
import {
BasePageComponentProps,
generateStaticParamsImplementation,
Expand Down
2 changes: 1 addition & 1 deletion tests/fixtures/use-cache/app/default/use-cache-component/static/ttl-1year/[slug]/page.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { unstable_cacheLife as cacheLife, unstable_cacheTag as cacheTag } from 'next/cache'
import { cacheLife, cacheTag } from '../../../../../next-cache'
import {
BasePageComponentProps,
generateStaticParamsImplementation,
Expand Down
2 changes: 1 addition & 1 deletion tests/fixtures/use-cache/app/default/use-cache-data/dynamic/ttl-1year/[slug]/page.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { unstable_cacheLife as cacheLife, unstable_cacheTag as cacheTag } from 'next/cache'
import { cacheLife, cacheTag } from '../../../../../next-cache'
import {
BasePageComponentProps,
getDataImplementation,
Expand Down
2 changes: 1 addition & 1 deletion tests/fixtures/use-cache/app/default/use-cache-data/dynamic/ttl-5seconds/[slug]/page.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { unstable_cacheLife as cacheLife, unstable_cacheTag as cacheTag } from 'next/cache'
import { cacheLife, cacheTag } from '../../../../../next-cache'
import {
BasePageComponentProps,
getDataImplementation,
Expand Down
2 changes: 1 addition & 1 deletion tests/fixtures/use-cache/app/default/use-cache-data/static/ttl-10seconds/[slug]/page.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { unstable_cacheLife as cacheLife, unstable_cacheTag as cacheTag } from 'next/cache'
import { cacheLife, cacheTag } from '../../../../../next-cache'
import {
BasePageComponentProps,
generateStaticParamsImplementation,
Expand Down
2 changes: 1 addition & 1 deletion tests/fixtures/use-cache/app/default/use-cache-data/static/ttl-1year/[slug]/page.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { unstable_cacheLife as cacheLife, unstable_cacheTag as cacheTag } from 'next/cache'
import { cacheLife, cacheTag } from '../../../../../next-cache'
import {
BasePageComponentProps,
generateStaticParamsImplementation,
Expand Down
2 changes: 1 addition & 1 deletion tests/fixtures/use-cache/app/default/use-cache-page/dynamic/ttl-1year/[slug]/page.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { unstable_cacheLife as cacheLife, unstable_cacheTag as cacheTag } from 'next/cache'
import { cacheLife, cacheTag } from '../../../../../next-cache'
import {
BasePageComponentProps,
getDataImplementation,
Expand Down
2 changes: 1 addition & 1 deletion tests/fixtures/use-cache/app/default/use-cache-page/dynamic/ttl-5seconds/[slug]/page.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { unstable_cacheLife as cacheLife, unstable_cacheTag as cacheTag } from 'next/cache'
import { cacheLife, cacheTag } from '../../../../../next-cache'
import {
BasePageComponentProps,
getDataImplementation,
Expand Down
2 changes: 1 addition & 1 deletion tests/fixtures/use-cache/app/default/use-cache-page/static/ttl-10seconds/[slug]/page.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { unstable_cacheLife as cacheLife, unstable_cacheTag as cacheTag } from 'next/cache'
import { cacheLife, cacheTag } from '../../../../../next-cache'
import {
BasePageComponentProps,
generateStaticParamsImplementation,
Expand Down
2 changes: 1 addition & 1 deletion tests/fixtures/use-cache/app/default/use-cache-page/static/ttl-1year/[slug]/page.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { unstable_cacheLife as cacheLife, unstable_cacheTag as cacheTag } from 'next/cache'
import { cacheLife, cacheTag } from '../../../../../next-cache'
import {
BasePageComponentProps,
generateStaticParamsImplementation,
Expand Down
20 changes: 20 additions & 0 deletions tests/fixtures/use-cache/app/next-cache.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
import * as NextCacheTyped from 'next/cache'

const NextCache = NextCacheTyped as any

export const cacheLife: any =
'cacheLife' in NextCache
? NextCache.cacheLife
: 'unstable_cacheLife' in NextCache
? NextCache.unstable_cacheLife
: () => {
throw new Error('both unstable_cacheLife and cacheLife are missing from next/cache')
}
export const cacheTag: any =
'cacheTag' in NextCache
? NextCache.cacheTag
: 'unstable_cacheTag' in NextCache
? NextCache.unstable_cacheTag
: () => {
throw new Error('both unstable_cacheTag and cacheTag are missing from next/cache')
}
6 changes: 6 additions & 0 deletions tests/utils/create-e2e-fixture.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -542,12 +542,18 @@ export const fixtureFactories = {
packagePath: 'apps/next-app',
buildCommand: 'nx run next-app:build',
publishDirectory: 'dist/apps/next-app/.next',
env: {
NX_ISOLATE_PLUGINS: 'false',
},
}),
nxIntegratedDistDir: () =>
createE2EFixture('nx-integrated', {
packagePath: 'apps/custom-dist-dir',
buildCommand: 'nx run custom-dist-dir:build',
publishDirectory: 'dist/apps/custom-dist-dir/dist',
env: {
NX_ISOLATE_PLUGINS: 'false',
},
Comment on lines +545 to +556
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nx on next canary starting failing with errors mentioned in nrwl/nx#27040 (but it's not exactly the same, because issue mention postinstall, while for us errors were happening at next build time, so despite that issue being closed with a fix, I think we are hitting similar problem in different place). I tried NX_ISOLATE_PLUGINS which was suggested as workaround there and that seemed to work here as well

}),
cliBeforeRegionalBlobsSupport: () =>
createE2EFixture('cli-before-regional-blobs-support', {
Expand Down
41 changes: 41 additions & 0 deletions tools/fetch-retry.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
// We are seeing quite a bit of 'fetch failed' cases in Github Actions that don't really reproduce
// locally. We are likely hitting some limits there when attempting to parallelize. They are not consistent
// so instead of reducing parallelism, we add a retry with backoff here.

const originalFetch = globalThis.fetch

const NUM_RETRIES = 5

globalThis.fetch = async (...args) => {
let backoff = 100
for (let attempt = 1; attempt <= NUM_RETRIES; attempt++) {
try {
return await originalFetch.apply(globalThis, args)
} catch (error) {
let shouldRetry = false
// not ideal, but there is no error code for that
if (error.message === 'fetch failed' && attempt < NUM_RETRIES) {
Copy link
Contributor Author

@pieh pieh Oct 16, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is happening very randomly and does not seem related to any specific fetch target:

  • it might be happening in fetch that CLI is doing to fetch extensions and result in confusing error about site not being linked (even tho we specifically pass site_id to cli command in our e2e tests)
  • it might be happening in data fetches in fixtures (both when preparing fixtures for integration tests and when building and deploying for e2e tests)

it seems more related to runners and hitting potential limits there

PS. I do hate it, but have no better idea as this would have to be replicated in a lot of places, some of which we don't control

// on this error we try again
shouldRetry = true
}

if (shouldRetry) {
// leave some trace in logs what's happening
console.error('[fetch-retry] fetch thrown, retrying...', {
args,
attempt,
errorMsg: error.message,
})

const currentBackoff = backoff
await new Promise((resolve) => {
setTimeout(resolve, currentBackoff)
})
backoff *= 2
continue
}

throw error
}
}
}
Loading