Serve the browser suite from PHP, not from the Symfony CLI

103 Playwright tests failed on CI and none of them were broken. The
Symfony CLI detects running Docker containers and injects its own
environment into the app it serves, overriding DATABASE_URL even when one
is explicitly set. On CI the Postgres service is a container, so the URL
the workflow sets was replaced with credentials nothing accepts, the app
could authenticate nobody, the per-worker login fixture died, and every
test in that worker fell over behind it. PHPUnit and every bin/console
step passed in the same job, because none of them go through the CLI.

    DATABASE_URL=postgresql://app:app@127.0.0.1:5432/app \
      symfony php -r 'echo getenv("DATABASE_URL");'

SYMFONY_SKIP_DOCKER_COMPOSE, SYMFONY_SKIP_DOCKER_SERVICES and
SYMFONY_DOCKER_ENV were all tried; none of them stop it, and there is no
--no-docker flag. So the suite now starts PHP's own server, which has no
detection to disable. The CLI install goes with it: it was there for
`symfony serve` alone.

Two things that server needs and does not do by itself. Naming
public/index.php as the router makes index.php answer everything,
including the 56 compiled assets /login pulls in — each one a kernel boot
returning 404 — so tests/e2e/support/router.php returns false for real
files and lets the server serve them off disk. And the cli-server SAPI
builds $_SERVER from the HTTP request and stops: getenv() answers,
$_SERVER does not, and Dotenv reads $_SERVER. Without the bridge in that
router, DATABASE_URL would silently revert to the committed default in
.env — the same failure as above arriving by a different road. HTTP_* is
excluded from the bridge, because $_SERVER['HTTP_PROXY'] is
indistinguishable from a Proxy header once a Request is built.

The server's log is redirected to a file and uploaded with the report
rather than piped to the console: it writes three lines per request, and
one slice of the suite produced 67,709 of them, which is how a real
failure gets buried. It is still the only place a PHP fatal that never
reached the kernel is recorded, so it is kept rather than silenced.

One onboarding test now retries opening the user menu. The menu is opened
by a Stimulus controller, and Playwright's actionability checks say
nothing about whether a controller has connected — the button is visible
in the server's HTML long before the module graph that animates it
arrives, and a click that lands in that gap is simply lost. FrankenPHP
served those 56 assets fast enough to hide it; this server does not.

Verified against a CI-shaped stack — host PHP, Postgres in a container,
the workflow's env block, this config starting the server: 185 passed,
14 skipped, 0 failed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

.github/workflows/e2e.yml

@@ -64,20 +64,16 @@ jobs:
             # 25M/60M, and the compose suite attaches a 3MB file — but CI never
             # builds that image, so without this the test cannot pass.
             #
-            # Two destinations, because two SAPIs are in play. `symfony serve`
-            # switches to PHP-FPM the moment it finds public/index.php, so the
-            # requests under test are served by FPM — and the CLI scan dir that
-            # `php -r PHP_CONFIG_FILE_SCAN_DIR` reports is NOT the one FPM
-            # reads. Symfony CLI's documented per-project override is a php.ini
-            # at the project root, and that is the copy the browser suite
-            # depends on. The scan-dir copy is for the console steps below,
-            # which run through the CLI — that SAPI deliberately ignores a
-            # php.ini in the working directory, so it needs its own.
+            # One destination, because there is now one SAPI: the console steps
+            # below and the server Playwright starts are both CLI. That SAPI
+            # ignores a php.ini in the working directory, so the scan dir is the
+            # only place it will be read from. This used to write a second copy
+            # to the project root as well, for the PHP-FPM process `symfony
+            # serve` forked off — see playwright.config.ts for why that server
+            # is gone.
             - name: Apply the app's PHP configuration
-              # sudo for the scan dir only; it is root-owned on the runner.
-              run: |
-                  cp frankenphp/conf.d/10-app.ini php.ini
-                  sudo cp frankenphp/conf.d/10-app.ini "$(php -r 'echo PHP_CONFIG_FILE_SCAN_DIR;')/"
+              # sudo because the scan dir is root-owned on the runner.
+              run: sudo cp frankenphp/conf.d/10-app.ini "$(php -r 'echo PHP_CONFIG_FILE_SCAN_DIR;')/"
 
             - name: Get Composer cache directory
               id: composer-cache
@@ -93,8 +89,13 @@ jobs:
             - name: Install PHP dependencies
               run: composer install --no-interaction --prefer-dist --no-progress
 
-            - name: Install Symfony CLI
-              run: curl -sS https://get.symfony.com/cli/installer | bash && sudo mv ~/.symfony*/bin/symfony /usr/local/bin/symfony
+            # No Symfony CLI step. It was installed for one thing — `symfony
+            # serve` — and that binary is the reason 103 browser tests failed:
+            # it sees the Postgres service container, decides it knows better,
+            # and overwrites the DATABASE_URL set above with credentials the app
+            # cannot use. Everything else here is plain `php bin/console`, which
+            # is why the console steps and PHPUnit always passed in the same job
+            # the browser suite was drowning in.
 
             - name: Setup Node
               uses: actions/setup-node@v4
@@ -180,10 +181,16 @@ jobs:
             - name: Run Playwright tests
               run: npx playwright test
 
+            # The web server's own log goes up with the report. It is kept out
+            # of the console because it is ~400 lines per page load, but it is
+            # the only place a PHP fatal that never reached the kernel is
+            # recorded — a trace cannot show what was never a response.
             - name: Upload Playwright report
               if: ${{ !cancelled() }}
               uses: actions/upload-artifact@v4
               with:
                   name: playwright-report
-                  path: playwright-report/
+                  path: |
+                      playwright-report/
+                      var/log/e2e-server.log
                   retention-days: 14

playwright.config.ts

@@ -6,7 +6,7 @@ import { defineConfig, devices } from "@playwright/test";
  * Two ways to run:
  *
  *   npm run test:e2e:docker  — against the compose.test.yaml stack (local default)
- *   npm run test:e2e         — against a Symfony local server this config starts (CI)
+ *   npm run test:e2e         — against a PHP built-in server this config starts (CI)
  *
  * Every worker owns a dedicated user and signs in once for itself, through a
  * worker-scoped fixture in tests/e2e/support/test.ts. That is why specs import
@@ -104,19 +104,57 @@ export default defineConfig({
     },
   ],
 
-  // Only when we are expected to boot the app ourselves. `symfony serve` does
-  // not exist in the container image, and pointing E2E_BASE_URL elsewhere while
-  // leaving this set would still race a second server onto the same fixtures.
+  // Only when we are expected to boot the app ourselves: pointing E2E_BASE_URL
+  // at a stack somebody else started, and leaving this set anyway, would race a
+  // second server onto the same fixtures.
   webServer: EXTERNAL_APP
     ? undefined
     : {
-        command: "symfony serve --port=8000 --no-tls --allow-http",
+        // PHP's own server, NOT `symfony serve`, and this is not a preference.
+        //
+        // The Symfony CLI detects running Docker containers and injects its own
+        // environment into the app it serves — including DATABASE_URL, which it
+        // OVERRIDES even when one is already set. Prove it with:
+        //
+        //   DATABASE_URL=postgresql://app:app@127.0.0.1:5432/app \
+        //     symfony php -r 'echo getenv("DATABASE_URL");'
+        //
+        // On CI the Postgres service is a Docker container, so the workflow set
+        // DATABASE_URL, `symfony serve` replaced it with credentials derived
+        // from that container, and the app could not authenticate anybody. The
+        // per-worker login fixture failed, and 103 tests failed behind it — a
+        // cascade whose only real cause was one environment variable. PHPUnit
+        // passed in the same job, because it never goes through the CLI.
+        //
+        // SYMFONY_SKIP_DOCKER_COMPOSE, SYMFONY_SKIP_DOCKER_SERVICES and
+        // SYMFONY_DOCKER_ENV were all tried and none of them stop it; there is
+        // no --no-docker flag. The built-in server has no detection to disable
+        // and inherits the environment exactly as given, which is the property
+        // that matters here.
+        //
+        // The router is tests/e2e/support/router.php and NOT public/index.php,
+        // which would answer every asset request with a kernel boot and a 404 —
+        // that file explains the rest.
+        //
+        // PHP_CLI_SERVER_WORKERS because that server is single-threaded
+        // otherwise, and a browser opening a page requests dozens of assets at
+        // once.
+        // Redirected, not silenced. This server logs a request line plus an
+        // Accepted and a Closing to stderr for every one of the ~60 requests a
+        // page makes, and Playwright pipes stderr straight into the console:
+        // one non-calendar slice of the suite produced 67,709 of those lines,
+        // which is how a genuine failure gets buried on CI. The file is kept
+        // and uploaded alongside the report, so a PHP fatal that never reached
+        // the kernel — and so never reached a trace — is still readable.
+        command:
+          "php -S 127.0.0.1:8000 -t public tests/e2e/support/router.php 2>var/log/e2e-server.log",
         url: `${BASE_URL}/login`,
         reuseExistingServer: !process.env.CI,
         timeout: 120_000,
         env: {
           APP_ENV: "test",
           APP_DEBUG: "1",
+          PHP_CLI_SERVER_WORKERS: "4",
         },
       },
 });

tests/e2e/onboarding.spec.ts

@@ -169,7 +169,22 @@ test("the user menu opens it again once it has been finished", async ({ page })
 
     await expect(backdrop(page)).toBeHidden();
 
-    await page.locator("#user-menu-btn").click();
+    // Retried, because the first click can be lost. The menu is opened by a
+    // Stimulus controller, and Playwright's actionability checks say nothing
+    // about whether a controller has connected — the button is visible and
+    // enabled in the server's HTML, long before the module graph that animates
+    // it has been fetched. A click that lands in that gap does nothing at all,
+    // and no amount of waiting afterwards opens a menu, so the retry has to
+    // wrap the click rather than follow it.
+    //
+    // It only ever showed up on a slow server: this page pulls 56 assets, and
+    // FrankenPHP serves them fast enough that the gap has usually closed by the
+    // time the test gets here. PHP's built-in server does not.
+    await expect(async () => {
+        await page.locator("#user-menu-btn").click();
+        await expect(page.locator("#user-menu")).toBeVisible({ timeout: 1_000 });
+    }).toPass();
+
     await page.locator("#user-menu-rerun-setup").click();
 
     await expect(wizard(page)).toBeVisible();

tests/e2e/support/router.php

@@ -0,0 +1,74 @@
+<?php
+
+declare(strict_types=1);
+
+/**
+ * The front controller PHP's built-in server uses for the browser suite.
+ *
+ * `php -S … -t public public/index.php` looks like it would do, and it does
+ * answer /login — but naming index.php as the router means index.php answers
+ * EVERYTHING, including the 56 compiled asset files /login pulls in. Each one
+ * boots a Symfony kernel and gets a 404 back. Returning false instead hands the
+ * request back to the server, which serves the file off disk with the right
+ * content type and no PHP at all.
+ *
+ * Two things are then fixed up before index.php is included, because the server
+ * has pointed both at this file:
+ *
+ *   SCRIPT_NAME     — Symfony derives the base URL from it, so every generated
+ *                     link would be prefixed with /tests/e2e/support/router.php
+ *   SCRIPT_FILENAME — read by the Runtime component and by the profiler
+ *
+ * Not in public/, deliberately: anything in there is web-reachable in a real
+ * deployment, and a second front controller that nobody audits is not something
+ * to ship for the sake of a test server.
+ *
+ * @see playwright.config.ts, which explains why the server is PHP's own rather
+ *      than `symfony serve`
+ */
+
+$publicDir = dirname(__DIR__, 3) . '/public';
+
+$path = parse_url((string) ($_SERVER['REQUEST_URI'] ?? '/'), PHP_URL_PATH);
+$path = \is_string($path) ? urldecode($path) : '/';
+
+// str_contains rather than a realpath check below: the server has already
+// normalised the request path, so a '..' surviving to here is an encoded one,
+// which is an attempt rather than an accident. This binds to 127.0.0.1 and
+// serves a test fixture database, but "test-only" is exactly the argument that
+// puts a traversal into a real deployment later.
+if ('/' !== $path && !str_contains($path, '..') && is_file($publicDir . $path)) {
+    return false;
+}
+
+$_SERVER['SCRIPT_NAME'] = '/index.php';
+$_SERVER['SCRIPT_FILENAME'] = $publicDir . '/index.php';
+
+// The environment, which the cli-server SAPI does not hand over.
+//
+// It builds $_SERVER from the HTTP request and stops there: getenv() answers,
+// $_SERVER and $_ENV do not. Symfony's Dotenv reads $_SERVER — populate() skips
+// a name only when `isset($_SERVER[$name])` — so without this every variable
+// the server was started with loses to the committed default in .env. On CI
+// that means DATABASE_URL silently reverts to `app:!ChangeMe!@127.0.0.1`, which
+// is the failure this whole file was written to get away from, arriving by a
+// different road. APP_ENV goes the same way, and the suite would quietly run
+// against the dev environment.
+//
+// ??= rather than assignment: REQUEST_URI and friends are already in there and
+// describe this request, not the process.
+foreach (getenv() as $name => $value) {
+    // HTTP_ is excluded for the reason Dotenv excludes it — $_SERVER['HTTP_X']
+    // is indistinguishable from an X header once a Request is built, so an
+    // ambient HTTP_PROXY would arrive as a client-supplied Proxy header. That
+    // is httpoxy (CVE-2016-5385), and it is cheaper to not open than to argue
+    // about whether a test server can be reached.
+    if (str_starts_with($name, 'HTTP_')) {
+        continue;
+    }
+
+    $_SERVER[$name] ??= $value;
+    $_ENV[$name] ??= $value;
+}
+
+require $publicDir . '/index.php';