E2E — auto-generated route manifests

In an admin app with dozens of pages and hundreds of API routes, the most common E2E mistake is forgetting to add a new route to the spec. Instead of hand-maintaining spec files, derive a route manifest from the filesystem and let a single spec iterate over all of them.

1. Why auto-generate

• No misses — new page / route flows through by regenerating
• Drift detection in CI — CI checks the manifest matches the code
• One spec covers everything — a single for-each spec is easier to review and maintain

Especially natural for frameworks where the filesystem is the router (Next.js App Router).

2. Page manifest from page.tsx

import fg from 'fast-glob';
import { writeFileSync } from 'node:fs';

const files = await fg(['src/app/**/page.tsx', '!src/app/api/**']);

const routes = files.map((f) => {
  return '/' + f
    .replace(/^src\/app\//, '')
    .replace(/\/page\.tsx$/, '')
    .replace(/\/\([^)]+\)/g, '')
    .replace(/\/\[(\.{3})?([^\]]+)\]/g, '/:$2')
    .replace(/\/_[^/]+/g, '');
});

writeFileSync('e2e/pages/manifest.json',
  JSON.stringify({ routes: routes.sort() }, null, 2));

Handles Next.js segments: [id] → :id, route groups (group), private folders _*.

3. API manifest from route.ts + methods

const files = await fg('src/app/api/**/route.ts');

const endpoints: Array<{ path: string; methods: string[] }> = [];

for (const file of files) {
  const src = readFileSync(file, 'utf-8');
  const methods = ['GET', 'POST', 'PUT', 'PATCH', 'DELETE']
    .filter((m) => new RegExp(`export\\s+async\\s+function\\s+${m}\\b`).test(src));
  if (methods.length === 0) continue;

  const path = '/' + file
    .replace(/^src\/app\//, '')
    .replace(/\/route\.ts$/, '')
    .replace(/\/\[(\.{3})?([^\]]+)\]/g, '/:$2');

  endpoints.push({ path, methods });
}

Only exported methods count.

4. Single spec iterates the manifest

import manifest from './manifest.json';

const TOLERATE_5XX = new Set([
  '/admin/crawl/products',
  '/admin/documents/edit/:id',
]);

for (const route of manifest.routes) {
  test(`page ${route} smoke`, async ({ page }) => {
    const url = route
      .replace(/:id/g, '00000000-0000-0000-0000-000000000000')
      .replace(/:slug/g, 'test-slug');

    const resp = await page.goto(url);
    const status = resp?.status() ?? 0;

    if (TOLERATE_5XX.has(route)) {
      expect(status).toBeGreaterThanOrEqual(200);
      return;
    }

    expect(status).toBeLessThan(500);
    await expect(page.locator('body')).toBeVisible();
  });
}

Each iteration becomes its own test case in the report.

5. Protect write methods

Running smoke tests against all routes would delete real data. Two layers:

• Tag write tests with @write
• Skip in PROD via Playwright grepInvert: /@write/
• Runtime guard helper skipWriteOnProd() as first line of any write test

test('delete user @write', async ({ request }) => {
  skipWriteOnProd();
  // ...
});

Two overlapping layers mean one mistake does not cause data loss.

6. Drift detection in CI

Commit the manifest. CI regenerates and diffs.

- name: Regenerate E2E manifests
  run: |
    pnpm tsx e2e/pages/generate-manifest.ts
    pnpm tsx e2e/equivalence/generate-manifest.ts
    git diff --exit-code -- e2e/pages/manifest.json e2e/equivalence/manifest.json

Commits that add a page but forget to regenerate fail CI.

7. Dummy values for dynamic segments

• UUID positions: 00000000-0000-0000-0000-000000000000
• Integer positions: 1
• Slug positions: test-slug
• Catch-all: empty or a/b/c

When dummies cause 404 / 500 that is not a product bug, add to TOLERATE_5XX, or seed a valid record and substitute.

8. Gotchas

Incomplete regex conversions — leaving (auth) in the URL yields 404s.

Timeout on heavy pages — large Server Component fetches hit the default test timeout. Override per route or shrink seed data.

Auth-protected routes 401 — all admin routes redirecting to login defeats the smoke. Seed storageState with a logged-in session.

Parallel write interference — two tests mutating the same row collide. Use test.describe.configure({ mode: 'serial' }) or per-domain workers.

Manifest diff noise — CRLF vs LF, indentation. Normalize JSON with indent: 2 + LF.

9. Growth path

• Stage 1 — smoke status < 500
• Stage 2 — check <main> / <h1> structural elements
• Stage 3 — generate from typed routes / OpenAPI, extend to response-schema checks
• Stage 4 — axe-core accessibility + Lighthouse per route

Stage 1 alone removes most "new page white-screen regressions".

Closing

The larger the admin app, the more the manifest-driven approach pays off. Humans reliably forget to add specs; machines reliably don't. Freeze "manifests are machine-made" as a rule and your specs stay thin and obvious to review.

Next

• testcontainers
• vitest-pytest-infra

References: Playwright · fast-glob · axe-core · Next.js App Router.