Two Browsers, One Document: How We Test Realtime Collaboration With Playwright
Our single-user E2E suite stayed green while collab bugs shipped. Here’s the small multi-user setup that finally caught them.
Three bugs hurt us more than any others last quarter. An edit vanished after a reconnect. A comment ended up anchored to the wrong paragraph after someone else typed above it. And the “Saved” badge kept saying “Saved” while the laptop was offline.
Our end-to-end suite was green for all three. Every run, every PR. It wasn’t broken. It just never tested the situation where the bugs live: two people in the same document at the same time.
So we built a small multi-user setup on top of Playwright. It’s maybe 150 lines of helpers, and it’s now the most valuable part of our E2E suite. Here’s how it works, where it bit us, and where I’d tell you not to bother.
Why one page per test misses this class of bug
A single-user test has one editor, one websocket, and one source of truth. If you type “hello”, you see “hello”. The sync layer is technically running, but nothing ever disagrees with it.
Collab bugs are disagreements. Two replicas diverge for a moment, then have to converge, and the bug is in how they do it. An offline edit gets dropped when the provider reconnects. A comment anchor resolves against a stale position. A status indicator reflects “request sent” rather than “other side has it”.
You need a second participant to produce any of that. Without one, you’re testing the editor, not the collaboration.
Contexts, not tabs
My first attempt opened two tabs in the same browser context. It looked like it worked, and it was wrong.
Tabs in one context share cookies, localStorage and IndexedDB. If you log in as Alice, Bob’s tab is also Alice. Worse, anything your app caches locally (offline state, draft storage, a persisted provider) can leak between the “two users” in ways real users never experience. Real collaborators are on different machines.
browser.newContext() gives you a fresh, isolated browser profile each time. Two contexts means two users with separate storage, separate cookies and separate network state. That last one matters later, because setOffline works per context.
For login, we do it once in a setup step and save each user’s state:
await context.storageState({ path: 'e2e/.auth/alice.json' });Then each test creates contexts with browser.newContext({ storageState: 'e2e/.auth/alice.json' }). No logging in through the UI before every test. One thing I'd flag: I pass baseURL explicitly when I create contexts by hand, rather than trusting that config options carry over. It costs one line and removes a question.
A fixture that hands you alice and bob
The thing that made this pleasant to write was a custom fixture. A test shouldn’t care how two users get into a document. It should just receive them, already there.
Ours creates a fresh document through a test-only API (gated behind a secret header, and only enabled outside production), opens two contexts as two different users, loads the same document in both, and waits until both report a live connection:
// e2e/fixtures.ts
import { test as base, expect, Page } from '@playwright/test';
import { randomUUID } from 'crypto';
type Collab = { alice: Page; bob: Page; docId: string };
const baseURL = process.env.E2E_BASE_URL ?? 'http://localhost:3000';
const headers = { 'x-e2e-key': process.env.E2E_KEY! };
export const test = base.extend<{ collab: Collab }>({
collab: async ({ browser, request }, use) => {
const res = await request.post('/api/test/documents', {
headers,
data: {
id: `e2e-${randomUUID()}`,
seed: ['First paragraph.', 'Second paragraph.', 'Third paragraph.'],
},
});
const { id: docId } = await res.json();
const aliceCtx = await browser.newContext({ baseURL, storageState: 'e2e/.auth/alice.json' });
const bobCtx = await browser.newContext({ baseURL, storageState: 'e2e/.auth/bob.json' });
const alice = await aliceCtx.newPage();
const bob = await bobCtx.newPage();
for (const page of [alice, bob]) {
await page.goto(`/docs/${docId}`);
await expect(page.getByTestId('connection-status')).toHaveText('Connected');
}
await use({ alice, bob, docId });
await aliceCtx.close();
await bobCtx.close();
await request.delete(`/api/test/documents/${docId}`, { headers });
},
});Two details here earned their place. The document is seeded with known paragraphs, so every test starts from the same shape. And the fixture waits for the connection indicator before handing anything over. That second one is a whole gotcha on its own, which I’ll get to.
Assert on state, not on sleeps
Here’s the pattern for “Alice types, Bob sees it”:
test("bob sees alice's edit", async ({ collab: { alice, bob } }) => {
await alice.locator('.ProseMirror').click();
await alice.keyboard.press('Control+End');
await alice.keyboard.type(' Alice was here.');
await expect(bob.locator('.ProseMirror')).toContainText('Alice was here.');
});No waitForTimeout. Web-first assertions like toContainText and toHaveText retry until they pass or time out, so you wait exactly as long as the sync takes and no longer. A sleep is either too short (flaky) or too long (slow), and usually both on different days.
For anything that isn’t a locator, like “do these two documents match?”, use expect.poll. It re-runs your async function until the matcher is satisfied. You'll see it in a moment.
One more thing about what to assert on. DOM text is fine for “did the words arrive”. It’s a poor judge of “are the documents identical”, because two different document structures can render the same text. For convergence, I want the editor’s actual state. So we expose a test-only hook, enabled only in E2E builds:
// components/Editor.tsx
useEffect(() => {
if (process.env.NEXT_PUBLIC_E2E !== '1' || !editor) return;
(window as any).__e2e = { getJSON: () => editor.getJSON() };
return () => { delete (window as any).__e2e; };
}, [editor]);It’s a small compromise. Production code now knows tests exist. I’m fine with that, because the alternative is asserting on brittle DOM and missing structural divergence.
Concurrent edits and the offline test
This is the test that earns the whole setup. Both users type at the same moment, in different places, and we check that they end up with the same document:
const docJSON = (page: Page) => page.evaluate(() => (window as any).__e2e.getJSON());
test('concurrent edits converge', async ({ collab: { alice, bob } }) => {
await alice.locator('.ProseMirror').click();
await alice.keyboard.press('Control+Home');
await bob.locator('.ProseMirror').click();
await bob.keyboard.press('Control+End');
await Promise.all([
alice.keyboard.type('ALICE-EDIT ', { delay: 20 }),
bob.keyboard.type(' BOB-EDIT', { delay: 20 }),
]);
await expect.poll(async () => {
const [a, b] = await Promise.all([docJSON(alice), docJSON(bob)]);
return JSON.stringify(a) === JSON.stringify(b);
}, { timeout: 10_000 }).toBe(true);
for (const page of [alice, bob]) {
await expect(page.locator('.ProseMirror')).toContainText('ALICE-EDIT');
await expect(page.locator('.ProseMirror')).toContainText('BOB-EDIT');
}
});The two checks do different jobs. The poll proves both replicas agree. The toContainText pair proves neither edit was lost. You need both, because two replicas can agree perfectly on a document that silently dropped someone's work.
Get Faisal Mujtaba’s stories in your inbox
Join Medium for free to get updates from this writer.
The comment-anchoring bug got the same treatment: Alice comments on the second paragraph, Bob types a new paragraph above it at the same time, and we assert that the highlighted comment text is still the right paragraph in both pages.
Offline is the other big one. context.setOffline(true) takes one user's network away without touching the other:
test('offline edits sync after reconnect', async ({ collab: { alice, bob } }) => {
const aliceCtx = alice.context();
await aliceCtx.setOffline(true);
await expect(alice.getByTestId('connection-status')).toHaveText('Offline');
await alice.locator('.ProseMirror').click();
await alice.keyboard.press('Control+End');
await alice.keyboard.type(' typed while offline');
await expect(alice.getByTestId('save-badge')).not.toHaveText('Saved');
await aliceCtx.setOffline(false);
await expect(alice.getByTestId('connection-status')).toHaveText('Connected', { timeout: 15_000 });
await expect(bob.locator('.ProseMirror')).toContainText('typed while offline');
await expect(alice.getByTestId('save-badge')).toHaveText('Saved');
});That one test covers two of our three original bugs: the vanished edit and the lying badge.
Honest caveat: how setOffline interacts with an already-open websocket has varied across browsers and versions. Don't assume it severs your connection. Assert that your own UI actually shows "Offline" before you type, like the test above does. If it doesn't flip in your setup, check the current Playwright docs and issue tracker rather than trusting my memory.
Network chaos, carefully
page.route is the tool for delaying or failing HTTP requests, and it's solid for things like a slow save endpoint. For websocket traffic, there's a newer API, page.routeWebSocket, that I believe lets you intercept and proxy socket messages. I haven't built anything serious on it, so I won't pretend to give you a signature. Check the current docs for the exact shape before you design around it.
Until then, setOffline plus a slow HTTP endpoint covers most of what we needed.
Keeping it deterministic
Realtime tests flake when they share anything. Our rules are boring on purpose:
- Every test gets a brand-new document with a unique id.
- No test reads what another test wrote.
- Cleanup runs in the fixture teardown, so a failed test can’t leak a document into the next run.
- Test users are fixed and shared, but they only ever touch their own fresh documents.
To hunt flakes, we run the suite with --repeat-each (for example npx playwright test collab --repeat-each=20). If a test fails once in twenty, it's lying to you, and you want to find that out locally rather than on a Friday deploy. We keep trace: 'retain-on-failure' on, because a trace of two pages side by side is the fastest way to see which user was out of sync and when.
Gotchas that cost us days
Asserting before the socket connects. Our first version typed immediately after goto. Locally the connection won the race. In CI it didn't, and the first keystrokes sometimes went into a not-yet-synced document. The fix was the "Connected" wait in the fixture. If your app has no visible connection state, add one. Your users benefit too.
Local green, CI flaky. CI machines are slower and noisier, so every hidden timing assumption surfaces. Anything that passed only because we’d typed fast enough on a quick laptop got exposed. Fixed timeouts were never the answer. Waiting on real signals was.
Cursor and selection placement. click() on the editor drops the caret wherever the click lands, which varies with layout. We now click once and then use Control+Home or Control+End to put the caret somewhere predictable. On macOS the modifier differs, so check Playwright's keyboard docs for how to handle that if your team runs tests on both.
Cleanup. Early on we left test documents behind, and the list view slowly turned into a graveyard. Teardown deletes them now, and the test API refuses ids that don’t start with e2e-.
Speed. These tests are slower, since each one launches two contexts and waits for real sync. A test that would take a second or two single-user is more like several seconds here. So we kept the suite small: roughly ten tests, each targeting a specific failure we’ve actually seen. We did not convert the rest of the suite.
What belongs in unit tests instead
I don’t want to oversell this. Browser-level multi-user tests are a blunt instrument.
If your concern is “does this merge or position-mapping logic behave correctly for these sequences of operations”, test that directly, with two in-memory replicas and no browser. It’s faster, deterministic, and lets you run thousands of randomized operation orders. Our comment-anchor math has unit tests like that, and they catch most regressions.
What the browser tests add is the glue: the provider, the reconnect behavior, the UI state that reflects connection and saving. Unit tests can’t see a badge lying to a user. Browser tests can’t cheaply explore every operation ordering. You want both, with the weight on the cheap side.
What I’d try first
Start with one test: the concurrent-edit convergence test above. Build the two-user fixture, add the connection wait, expose the JSON hook, and run it with --repeat-each=20. That alone will tell you whether your setup is trustworthy, and it'll probably find something.
Add the offline test second. If your app shows any kind of saved or synced indicator, that test is where it gets caught lying.
When not to bother: if your product is single-user with an occasional shared link, or realtime is a side feature rather than the core, this is more setup than it’s worth. Same if you don’t yet have unit tests around your merge or anchoring logic. Write those first.
But if people can type in the same document, a green single-user suite is only telling you half the story.
