Files
app__bitwarden-clients/e2e/README.md
Oscar Hinton e3d0bc5709 Add SeederApi seeding to the e2e suite (#24381)
* Add SeederApi seeding to the e2e suite

Ports the seeder framework from the playwright branch into e2e/, updated
for the current SeederApi: basic auth, the { urls } emergency access
invite response, and the full SingleUserScene request and result.

Each test gets a play whose id tags what it seeds and what its pages
create, and which is deleted when the test ends. Seeding tests are
skipped when no SeederApi account is configured.

* Harden e2e seeding configuration and cleanup

- Skip seeding tests against a deployed vault while the SeederApi is
  local, since it seeds a database that vault never reads.
- Read E2E_SEEDER_USERNAME and E2E_SEEDER_PASSWORD as a pair, failing
  on a partial set, and resolve the seeder URL the same way whatever
  supplied the account. Empty values count as unset.
- Set up `play` before the browser context so the page closes before
  its data is deleted.
- Assert the seeded user is the one signed in, through the automation
  driver's lock capability.
- Document that client-created data is only cleaned up when Api and
  Identity have play-id tracking enabled.

* Use Playwright's request client and routing for e2e seeding

- Call the SeederApi through an APIRequestContext opened per play, so
  credentials are read once, calls appear in traces, and an unreachable
  SeederApi says so.
- Tag requests to the vault with the play id through context routing
  instead of patching fetch, which also covers XHR.
- Record video for newPage contexts, which the config's video option
  does not reach.
- Simplify Play.mangle.

* Say up front when seeding e2e tests skip and when they fail
2026-11-09 09:04:11 +01:00

4.7 KiB

E2e

NOTE: This package is currently a key-management team internal experiment. Please do not
yet contribute to it, or place your tests here. We may in the future standardize this via
an ADR at which point other teams may use it.

Playwright end-to-end suites for the web, browser extension and desktop clients, plus a shared suite, which can run tests involving multiple clients at the same time.

Install the browser once:

npx playwright install chromium
npm run test:e2e:web       # webpack dev server + chromium
npm run test:e2e:browser   # builds the chrome extension, loads it unpacked
npm run test:e2e:desktop   # builds electron main/renderer/preload, launches the app
npm run test:e2e:shared    # web + extension, extension + desktop, e.g. shared unlock
npm run test:e2e:all       # all four, sequentially

Failing tests keep a video and a trace (npx playwright show-trace <trace.zip>). Set E2E_VIDEO=1 to record every test; both land in e2e/test-results/<test>/.

Credentials

Accounts live in .debug/credentials.txt (gitignored), an INI-style file:

[local-web]
email=e2e-web@example.com
password=...
server=https://localhost:8080

Since these are on-disk credentials, only use test accounts.

server is the account's web vault URL, and the self-hosted base URL desktop and browser log into. A bare host means https://<host>.

Every suite defaults to local-web, because the web client's API URLs are baked into its build; desktop and browser point their self-hosted setting at it. Override with E2E_ACCOUNT=<section>.

Multi-account tests sign in a second account from local-web-2, on the same server. Override with E2E_SECOND_ACCOUNT=<section>.

The web suite builds and serves the client itself when server is the dev server, https://localhost:8080. Any other server must be a deployed vault:

E2E_ACCOUNT=usdev-e2e npm run test:e2e:web

E2E_EMAIL, E2E_PASSWORD and E2E_SERVER take precedence over the file when all three are set. CI uses them.

Seeded data

Tests that need their own users, organizations or items create them through the server's SeederApi (util/SeederApi in bitwarden/server) instead of the shared accounts above. Run it beside the local server:

cd util/SeederApi && dotnet run   # http://localhost:5047

Tests that seed are skipped, not failed, when no [seeder] account is configured, and when a deployed vault is paired with a local SeederApi. With an account configured, the SeederApi must be running: every test importing test from src/seeder/fixtures.ts fails without it, including ones that seed nothing.

It takes a basic-auth account from its seederSettings:accounts. Give the suite the same one in a [seeder] section:

[seeder]
username=e2e
password=...
url=http://localhost:5047

url is optional. E2E_SEEDER_USERNAME and E2E_SEEDER_PASSWORD take precedence over the file, as a pair, and E2E_SEEDER_URL over url.

Data a test creates through the client, rather than seeds, is only deleted when the server tracks it too. Add "testPlayIdTrackingEnabled": true to globalSettings in the server's dev/secrets.json, rerun dev/setup_secrets.ps1, and restart Api and Identity. The SeederApi enables it on its own.

Import test from src/seeder/fixtures.ts, which adds a play fixture holding the test's data, and newPage for a second user's page:

const user = await play.scene(
  new SingleUserScene({ email: "user@example.com", password: randomUUID() }),
);
await logInToWebVault(page, seededAccount(user));

The server mangles unique values such as emails; scene.mangle("user@example.com") returns the stored one. Everything a play seeds, and with tracking on everything its pages create through the client, is deleted when the test ends. E2E_KEEP_SEEDED_DATA=1 keeps it and prints the command that deletes it; the SeederApi also deletes play data older than a day.

Add a scene or query by subclassing SceneTemplate in src/seeder/scenes/ or QueryTemplate in src/seeder/queries/, named after the server's class and typed after its Request and result.

CI

.github/workflows/test-e2e-usdev.yml runs every suite against usdev nightly, and on a PR when it is labeled ci:km-e2e-tests. The web suite drives the deployed usdev vault, so a PR's web changes are not under test; the others build their client from the PR.

Headless

Only the web suite is headless. The others open a window:

  • Chromium's new headless mode is the only one that loads extensions, and it restarts the extension's service worker repeatedly.
  • Electron has no headless mode.

On a Linux CI runner, wrap them in a virtual display:

xvfb-run -a npm run test:e2e:browser
xvfb-run -a npm run test:e2e:desktop
xvfb-run -a npm run test:e2e:shared