Designing a Real-Time Collaborative Editing Feature
March 1, 20263 min read6 views
Designing a Real-Time Collaborative Editing Feature Like Notion
Building real-time collaboration isn't just about syncing keystrokes. It's about maintaining consistency across distributed clients, handling network partitions gracefully, showing presence without overwhelming the system, and doing all of this without data loss when two people edit the same sentence simultaneously.
This is a deep dive into the architecture behind collaborative editing: the theoretical foundations (CRDTs vs OT), practical implementation with Yjs, awareness protocols for cursor presence, and how to wire everything together in a production React application.
The Fundamental Problem: Distributed Consensus
┌─────────────────────────────────────────────────────────────────────────────┐
│ THE COLLABORATIVE EDITING PROBLEM │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ Initial state: "Hello World" │
│ │
│ ┌─────────────┐ ┌─────────────┐ │
│ │ User A │ │ User B │ │
│ │ │ │ │ │
│ │ Inserts "!" │ │ Deletes "o" │ │
│ │ at pos 11 │ │ at pos 7 │ │
│ └──────┬──────┘ └──────┬──────┘ │
│ │ │ │
│ │ Network latency │ │
│ │ ◄────────────────────────────────────────► │
│ │ │ │
│ ▼ ▼ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ User A sees: "Hello World!" (applied own change) │ │
│ │ User B sees: "Hello Wrld" (applied own change) │ │
│ │ │ │
│ │ Then they receive each other's changes... │ │
│ │ │ │
│ │ Naive application: │ │
│ │ User A: "Hello World!" + delete at pos 7 = "Hello Wrld!" │ │
│ │ User B: "Hello Wrld" + insert at pos 11 = "Hello Wrld!" ✓ │ │
│ │ │ │
│ │ Wait, they converged! But what if... │ │
│ │ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │
│ Problematic case: "Hello World" │
│ │
│ User A: Insert "X" at position 5 → "HelloX World" │
│ User B: Insert "Y" at position 5 → "HelloY World" │
│ │
│ After sync (naive): │
│ User A: "HelloX World" + insert Y at 5 = "HelloYX World" │
│ User B: "HelloY World" + insert X at 5 = "HelloXY World" │
│ │
│ DIVERGENCE! Documents are now permanently inconsistent. │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
Two approaches solve this: Operational Transformation (OT) and Conflict-free Replicated Data Types (CRDTs).
Operational Transformation (OT): The Classic Approach
OT was invented at Xerox PARC in 1989 and powers Google Docs. The core idea: transform operations against concurrent operations to preserve intent.
How OT Works
┌─────────────────────────────────────────────────────────────────────────────┐
│ OPERATIONAL TRANSFORMATION │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ Document: "abc" │
│ │
│ User A: Insert('X', 1) → "aXbc" │
│ User B: Insert('Y', 2) → "abYc" │
│ │
│ When User A receives User B's operation: │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ Original: Insert('Y', 2) │ │
│ │ │ │
│ │ Transform against Insert('X', 1): │ │
│ │ - Y's position (2) is AFTER X's position (1) │ │
│ │ - X shifted everything after position 1 by 1 │ │
│ │ - New operation: Insert('Y', 3) │ │
│ │ │ │
│ │ Apply: "aXbc" + Insert('Y', 3) = "aXbYc" │ │
│ │ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │
│ When User B receives User A's operation: │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ Original: Insert('X', 1) │ │
│ │ │ │
│ │ Transform against Insert('Y', 2): │ │
│ │ - X's position (1) is BEFORE Y's position (2) │ │
│ │ - No transformation needed │ │
│ │ - Operation stays: Insert('X', 1) │ │
│ │ │ │
│ │ Apply: "abYc" + Insert('X', 1) = "aXbYc" │ │
│ │ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │
│ Both converge to "aXbYc" ✓ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
The Transform Function
// Simplified OT transform for insert operations
type Operation =
| { type: 'insert'; position: number; char: string }
| { type: 'delete'; position: number };
function transform(op1: Operation, op2: Operation): Operation {
// op1 is the operation to transform
// op2 is the operation that was applied concurrently
if (op1.type === 'insert' && op2.type === 'insert') {
if (op1.position <= op2.position) {
return op1; // No change needed
} else {
return { ...op1, position: op1.position + 1 };
}
}
if (op1.type === 'insert' && op2.type === 'delete') {
if (op1.position <= op2.position) {
return op1;
} else {
return { ...op1, position: op1.position - 1 };
}
}
if (op1.type === 'delete' && op2.type === 'insert') {
if (op1.position < op2.position) {
return op1;
} else {
return { ...op1, position: op1.position + 1 };
}
}
if (op1.type === 'delete' && op2.type === 'delete') {
if (op1.position < op2.position) {
return op1;
} else if (op1.position > op2.position) {
return { ...op1, position: op1.position - 1 };
} else {
// Both deleted the same character - op1 becomes no-op
return { type: 'noop' } as any;
}
}
return op1;
}
OT's Achilles Heel: The Server
┌─────────────────────────────────────────────────────────────────────────────┐
│ OT ARCHITECTURE │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────┐ ┌─────────┐ ┌─────────┐ │
│ │ Client A│ │ Client B│ │ Client C│ │
│ └────┬────┘ └────┬────┘ └────┬────┘ │
│ │ │ │ │
│ │ Operations │ Operations │ │
│ └──────────────────┼──────────────────┘ │
│ │ │
│ ▼ │
│ ┌───────────────┐ │
│ │ │ │
│ │ SERVER │◄─── Central authority │
│ │ │ - Serializes operations │
│ │ - Transform │ - Maintains canonical state │
│ │ - Broadcast │ - Single point of failure │
│ │ │ │
│ └───────────────┘ │
│ │
│ Problems: │
│ 1. Server must process ALL operations in order │
│ 2. Offline editing is extremely complex │
│ 3. Transform functions get exponentially complex with operation types │
│ 4. Correctness proofs are notoriously difficult │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
OT requires a central server to serialize operations. This makes:
- Offline editing extremely difficult (you need to queue and transform potentially hundreds of operations)
- P2P collaboration impossible without a coordinator
- Implementation error-prone (Google's OT implementation is rumored to be millions of lines)
CRDTs: The Modern Approach
CRDTs (Conflict-free Replicated Data Types) take a fundamentally different approach: design data structures that mathematically guarantee convergence without coordination.
The CRDT Philosophy
┌─────────────────────────────────────────────────────────────────────────────┐
│ CRDT PRINCIPLES │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ 1. EVERY operation is commutative │
│ - Order doesn't matter: A + B = B + A │
│ │
│ 2. EVERY operation is idempotent │
│ - Applying twice has same effect: A + A = A │
│ │
│ 3. EVERY operation is associative │
│ - Grouping doesn't matter: (A + B) + C = A + (B + C) │
│ │
│ Result: No matter what order operations arrive, or how many times │
│ they're applied, all replicas converge to the same state. │
│ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ Client A ─────────────────────────────────────────► State X │ │
│ │ ops: [1, 2, 3] │ │
│ │ │ │
│ │ Client B ─────────────────────────────────────────► State X │ │
│ │ ops: [2, 1, 3] (different order!) │ │
│ │ │ │
│ │ Client C ─────────────────────────────────────────► State X │ │
│ │ ops: [3, 2, 1, 2] (different order + duplicate!) │ │
│ │ │ │
│ │ ALL converge to identical State X │ │
│ │ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
How Text CRDTs Work: Unique Character IDs
The key insight: instead of positions, assign unique, globally ordered identifiers to each character.
┌─────────────────────────────────────────────────────────────────────────────┐
│ TEXT CRDT: UNIQUE CHARACTER IDS │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ Traditional: "Hello" │
│ ┌───┬───┬───┬───┬───┐ │
│ │ H │ e │ l │ l │ o │ │
│ └───┴───┴───┴───┴───┘ │
│ 0 1 2 3 4 ◄── Position-based (problematic) │
│ │
│ CRDT: "Hello" │
│ ┌─────────────┬─────────────┬─────────────┬─────────────┬─────────────┐ │
│ │ H │ e │ l │ l │ o │ │
│ │ id: A.1 │ id: A.2 │ id: A.3 │ id: A.4 │ id: A.5 │ │
│ └─────────────┴─────────────┴─────────────┴─────────────┴─────────────┘ │
│ │
│ IDs are: │
│ - Globally unique (clientId + sequence number) │
│ - Totally ordered (can always determine which comes first) │
│ - Immutable (never change once assigned) │
│ │
│ Insert 'X' between 'e' and 'l': │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ New character gets ID that orders between A.2 and A.3 │ │
│ │ ID: B.1 (where B.1 > A.2 and B.1 < A.3 in the ordering) │ │
│ │ │ │
│ │ ┌───────┬───────┬───────┬───────┬───────┬───────┐ │ │
│ │ │ H │ e │ X │ l │ l │ o │ │ │
│ │ │ A.1 │ A.2 │ B.1 │ A.3 │ A.4 │ A.5 │ │ │
│ │ └───────┴───────┴───────┴───────┴───────┴───────┘ │ │
│ │ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │
│ Deletion = tombstone (mark as deleted, don't remove): │
│ ┌───────┬───────┬───────┬───────┬───────┬───────┐ │
│ │ H │ e │ X │ l │ l │ o │ │
│ │ A.1 │ A.2 │ B.1 ✝ │ A.3 │ A.4 │ A.5 │ │
│ └───────┴───────┴───────┴───────┴───────┴───────┘ │
│ │ │
│ └─► Tombstoned: excluded from visible text │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
Fractional Indexing: The ID Generation Strategy
// Simplified fractional indexing
// Real implementations use more sophisticated algorithms
type CharID = {
clientId: string;
clock: number;
position: number[]; // Fractional position as array
};
function generateIdBetween(
left: CharID | null,
right: CharID | null,
clientId: string,
clock: number
): CharID {
const leftPos = left?.position ?? [0];
const rightPos = right?.position ?? [1];
// Find a position between left and right
const newPos: number[] = [];
let i = 0;
while (true) {
const l = leftPos[i] ?? 0;
const r = rightPos[i] ?? 1;
if (r - l > 1) {
// Room to insert between
newPos.push(Math.floor((l + r) / 2));
break;
} else if (r - l === 1) {
// Need to go deeper
newPos.push(l);
i++;
// Continue with next level
} else {
// l === r, need to differentiate with next level
newPos.push(l);
i++;
}
}
return { clientId, clock, position: newPos };
}
// Comparison function
function compareIds(a: CharID, b: CharID): number {
// First compare by position
for (let i = 0; i < Math.max(a.position.length, b.position.length); i++) {
const aVal = a.position[i] ?? 0;
const bVal = b.position[i] ?? 0;
if (aVal !== bVal) return aVal - bVal;
}
// Tiebreaker: client ID
if (a.clientId !== b.clientId) {
return a.clientId < b.clientId ? -1 : 1;
}
// Final tiebreaker: clock
return a.clock - b.clock;
}
OT vs CRDT: When to Use What
┌─────────────────────────────────────────────────────────────────────────────┐
│ OT vs CRDT COMPARISON │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ Aspect │ OT │ CRDT │
│ ────────────────────┼───────────────────────┼───────────────────────── │
│ Architecture │ Central server │ Peer-to-peer possible │
│ │ required │ │
│ ────────────────────┼───────────────────────┼───────────────────────── │
│ Offline support │ Complex, limited │ Native, unlimited │
│ ────────────────────┼───────────────────────┼───────────────────────── │
│ Memory overhead │ Low (operations only) │ Higher (tombstones, IDs) │
│ ────────────────────┼───────────────────────┼───────────────────────── │
│ Implementation │ Transform functions │ Data structure design │
│ complexity │ are error-prone │ is mathematical │
│ ────────────────────┼───────────────────────┼───────────────────────── │
│ Consistency model │ Eventually consistent │ Strong eventual consistency │
│ │ (needs server) │ (guaranteed) │
│ ────────────────────┼───────────────────────┼───────────────────────── │
│ Undo/redo │ Straightforward │ Requires additional work │
│ ────────────────────┼───────────────────────┼───────────────────────── │
│ Real-world users │ Google Docs │ Figma, Notion, Linear │
│ │
│ Recommendation: │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ Use OT if: │ │
│ │ - You're Google and can afford millions of lines of code │ │
│ │ - Server-centric architecture is acceptable │ │
│ │ - You need perfect undo/redo semantics │ │
│ │ │ │
│ │ Use CRDTs if: │ │
│ │ - You need offline support │ │
│ │ - You want P2P collaboration │ │
│ │ - You want to use battle-tested libraries (Yjs, Automerge) │ │
│ │ - You're building from scratch │ │
│ │ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
For most teams building collaborative features today, CRDTs with Yjs is the right choice.
Yjs: The Production CRDT Library
Yjs is the most widely-used CRDT implementation. It powers Notion's collaboration, Linear, and many others.
Core Concepts
import * as Y from 'yjs';
// Create a Yjs document
const ydoc = new Y.Doc();
// Shared types (CRDT data structures)
const ytext = ydoc.getText('content'); // Collaborative text
const yarray = ydoc.getArray('items'); // Collaborative array
const ymap = ydoc.getMap('metadata'); // Collaborative object
// All operations are automatically conflict-free
ytext.insert(0, 'Hello'); // Insert at position 0
ytext.delete(0, 5); // Delete 5 characters from position 0
// Observe changes
ytext.observe((event) => {
console.log('Text changed:', event.delta);
// delta format: [{ retain: 5 }, { insert: 'World' }]
});
Document Structure for a Notion-like Editor
import * as Y from 'yjs';
// Document structure
interface BlockData {
id: string;
type: 'paragraph' | 'heading' | 'list' | 'code' | 'image';
content: Y.Text; // Rich text content
properties: Y.Map<any>; // Block-specific properties
children: Y.Array<string>; // Child block IDs (for nesting)
}
function createDocument(): Y.Doc {
const doc = new Y.Doc();
// Root array of block IDs (maintains order)
const blocks = doc.getArray<string>('blocks');
// Map of block ID -> block data
const blockMap = doc.getMap<Y.Map<any>>('blockMap');
return doc;
}
function createBlock(
doc: Y.Doc,
type: BlockData['type'],
parentId?: string
): string {
const blockId = crypto.randomUUID();
const blockMap = doc.getMap<Y.Map<any>>('blockMap');
// Create block data
const blockData = new Y.Map<any>();
blockData.set('id', blockId);
blockData.set('type', type);
blockData.set('content', new Y.Text());
blockData.set('properties', new Y.Map());
blockData.set('children', new Y.Array<string>());
// Add to block map
blockMap.set(blockId, blockData);
// Add to parent or root
if (parentId) {
const parent = blockMap.get(parentId);
const children = parent?.get('children') as Y.Array<string>;
children?.push([blockId]);
} else {
const rootBlocks = doc.getArray<string>('blocks');
rootBlocks.push([blockId]);
}
return blockId;
}
Rich Text with Formatting
import * as Y from 'yjs';
// Yjs supports rich text attributes natively
const ytext = new Y.Text();
// Insert with formatting
ytext.insert(0, 'Hello', { bold: true });
ytext.insert(5, ' World', { italic: true });
// Apply formatting to range
ytext.format(0, 5, { underline: true });
// Result: "Hello World"
// ^^^^^ bold + underline
// ^^^^^^ italic
// Delta representation
console.log(ytext.toDelta());
// [
// { insert: 'Hello', attributes: { bold: true, underline: true } },
// { insert: ' World', attributes: { italic: true } }
// ]
Awareness Protocol: Cursor Presence
Real-time collaboration needs more than just document sync. Users need to see each other's cursors, selections, and online status.
┌─────────────────────────────────────────────────────────────────────────────┐
│ AWARENESS PROTOCOL │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ Document State (CRDT) Awareness State (Ephemeral) │
│ ┌─────────────────────┐ ┌─────────────────────────────────────┐ │
│ │ │ │ │ │
│ │ Persisted │ │ NOT persisted │ │
│ │ Conflict-free │ │ Last-write-wins │ │
│ │ Synced via CRDT │ │ Synced via broadcast │ │
│ │ │ │ │ │
│ │ - Text content │ │ - Cursor position │ │
│ │ - Block structure │ │ - Selection range │ │
│ │ - Formatting │ │ - User name/color │ │
│ │ │ │ - Online status │ │
│ │ │ │ - "Currently editing" indicator │ │
│ │ │ │ │ │
│ └─────────────────────┘ └─────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
Implementing Awareness with Yjs
import * as Y from 'yjs';
import { Awareness } from 'y-protocols/awareness';
interface UserAwareness {
user: {
id: string;
name: string;
color: string;
avatar?: string;
};
cursor: {
anchor: number; // Selection start
head: number; // Selection end (cursor position)
blockId: string; // Which block the cursor is in
} | null;
lastActive: number;
isTyping: boolean;
}
function setupAwareness(doc: Y.Doc): Awareness {
const awareness = new Awareness(doc);
// Set local user state
awareness.setLocalState({
user: {
id: crypto.randomUUID(),
name: 'User ' + Math.floor(Math.random() * 1000),
color: getRandomColor(),
},
cursor: null,
lastActive: Date.now(),
isTyping: false,
} satisfies UserAwareness);
// Listen for changes to other users' awareness
awareness.on('change', ({ added, updated, removed }) => {
const states = awareness.getStates() as Map<number, UserAwareness>;
states.forEach((state, clientId) => {
if (clientId === doc.clientID) return; // Skip self
console.log('User state:', state);
// Render cursor, selection, presence indicator
});
});
return awareness;
}
// Update cursor position
function updateCursor(
awareness: Awareness,
blockId: string,
anchor: number,
head: number
) {
const state = awareness.getLocalState() as UserAwareness;
awareness.setLocalStateField('cursor', {
blockId,
anchor,
head,
});
awareness.setLocalStateField('lastActive', Date.now());
}
// Typing indicator
let typingTimeout: NodeJS.Timeout;
function setTyping(awareness: Awareness) {
awareness.setLocalStateField('isTyping', true);
clearTimeout(typingTimeout);
typingTimeout = setTimeout(() => {
awareness.setLocalStateField('isTyping', false);
}, 1000);
}
Rendering Cursors
// React component for rendering other users' cursors
import { useEffect, useState } from 'react';
import { Awareness } from 'y-protocols/awareness';
interface RemoteCursor {
clientId: number;
user: { name: string; color: string };
cursor: { blockId: string; anchor: number; head: number };
}
function useRemoteCursors(awareness: Awareness): RemoteCursor[] {
const [cursors, setCursors] = useState<RemoteCursor[]>([]);
useEffect(() => {
const updateCursors = () => {
const states = awareness.getStates() as Map<number, UserAwareness>;
const remoteCursors: RemoteCursor[] = [];
states.forEach((state, clientId) => {
// Skip self and users without cursors
if (clientId === awareness.doc.clientID) return;
if (!state.cursor) return;
remoteCursors.push({
clientId,
user: state.user,
cursor: state.cursor,
});
});
setCursors(remoteCursors);
};
awareness.on('change', updateCursors);
updateCursors();
return () => awareness.off('change', updateCursors);
}, [awareness]);
return cursors;
}
// Cursor component
function RemoteCursorOverlay({ cursors }: { cursors: RemoteCursor[] }) {
return (
<>
{cursors.map((cursor) => (
<div
key={cursor.clientId}
className="remote-cursor"
style={{
position: 'absolute',
// Position based on cursor.cursor coordinates
borderLeft: `2px solid ${cursor.user.color}`,
}}
>
{/* Cursor caret */}
<div
className="cursor-caret"
style={{ backgroundColor: cursor.user.color }}
/>
{/* User label */}
<div
className="cursor-label"
style={{ backgroundColor: cursor.user.color }}
>
{cursor.user.name}
</div>
{/* Selection highlight (if anchor !== head) */}
{cursor.cursor.anchor !== cursor.cursor.head && (
<div
className="selection-highlight"
style={{
backgroundColor: cursor.user.color,
opacity: 0.3,
}}
/>
)}
</div>
))}
</>
);
}
Backend Architecture
The backend needs to:
- Relay updates between clients (WebSocket)
- Persist document state
- Handle authentication and permissions
- Scale horizontally
┌─────────────────────────────────────────────────────────────────────────────┐
│ COLLABORATION BACKEND ARCHITECTURE │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ Clients │
│ ┌─────┐ ┌─────┐ ┌─────┐ │
│ │ A │ │ B │ │ C │ │
│ └──┬──┘ └──┬──┘ └──┬──┘ │
│ │ │ │ │
│ └────────┼────────┘ │
│ │ WebSocket │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ LOAD BALANCER │ │
│ │ (sticky sessions by docId) │ │
│ └───────────────────────────────┬─────────────────────────────────────┘ │
│ │ │
│ ┌───────────────────────┼───────────────────────┐ │
│ │ │ │ │
│ ▼ ▼ ▼ │
│ ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ │
│ │ WS Server │ │ WS Server │ │ WS Server │ │
│ │ (Node.js) │◄─────►│ (Node.js) │◄─────►│ (Node.js) │ │
│ │ │ │ │ │ │ │
│ │ Docs: A, B │ Redis │ Docs: C, D │ Redis │ Docs: E, F │ │
│ │ │ Pub │ │ Sub │ │ │
│ └───────┬───────┘ └───────┬───────┘ └───────┬───────┘ │
│ │ │ │ │
│ └───────────────────────┼───────────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ PERSISTENCE LAYER │ │
│ │ │ │
│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │ │
│ │ │ Redis │ │ PostgreSQL │ │ S3 / Object Store │ │ │
│ │ │ │ │ │ │ │ │ │
│ │ │ - Pub/Sub │ │ - Doc meta │ │ - Yjs state vectors │ │ │
│ │ │ - Doc locks │ │ - Versions │ │ - Snapshots │ │ │
│ │ │ - Presence │ │ - Audit log │ │ - Binary updates │ │ │
│ │ │ │ │ │ │ │ │ │
│ │ └─────────────┘ └─────────────┘ └─────────────────────┘ │ │
│ │ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
WebSocket Server with y-websocket
// server.ts
import { WebSocketServer } from 'ws';
import { setupWSConnection, docs } from 'y-websocket/bin/utils';
import * as Y from 'yjs';
const wss = new WebSocketServer({ port: 1234 });
// In-memory document storage (use persistence in production)
const documents = new Map<string, Y.Doc>();
wss.on('connection', (ws, req) => {
// Extract document ID from URL
const docId = req.url?.slice(1) || 'default';
// Auth check
const token = req.headers.authorization?.split(' ')[1];
if (!validateToken(token, docId)) {
ws.close(4401, 'Unauthorized');
return;
}
// Set up Yjs WebSocket handling
setupWSConnection(ws, req, {
docName: docId,
gc: true, // Enable garbage collection of deleted items
});
});
// Persistence callback (called when document updates)
import { LeveldbPersistence } from 'y-leveldb';
const persistence = new LeveldbPersistence('./yjs-docs');
// Or use custom persistence
async function persistDocument(docId: string, doc: Y.Doc) {
const state = Y.encodeStateAsUpdate(doc);
await saveToS3(docId, state);
}
Production-Ready Server with Hocuspocus
Hocuspocus is a production-ready Yjs WebSocket backend:
// server.ts
import { Hocuspocus } from '@hocuspocus/server';
import { Database } from '@hocuspocus/extension-database';
import { Redis } from '@hocuspocus/extension-redis';
import { Logger } from '@hocuspocus/extension-logger';
const server = new Hocuspocus({
port: 1234,
// Extensions
extensions: [
new Logger(),
// Redis for horizontal scaling
new Redis({
host: process.env.REDIS_HOST,
port: 6379,
}),
// Database persistence
new Database({
fetch: async ({ documentName }) => {
// Load document from database
const doc = await prisma.document.findUnique({
where: { id: documentName },
});
return doc?.data ?? null;
},
store: async ({ documentName, state }) => {
// Save document to database
await prisma.document.upsert({
where: { id: documentName },
update: { data: state, updatedAt: new Date() },
create: { id: documentName, data: state },
});
},
}),
],
// Authentication
async onAuthenticate({ token, documentName }) {
const user = await validateToken(token);
if (!user) {
throw new Error('Unauthorized');
}
const hasAccess = await checkDocumentAccess(user.id, documentName);
if (!hasAccess) {
throw new Error('Forbidden');
}
return { user };
},
// Document lifecycle
async onLoadDocument({ document, documentName, context }) {
// Called when document is loaded
console.log(`Document ${documentName} loaded by ${context.user.id}`);
},
async onStoreDocument({ document, documentName }) {
// Called periodically and on close
console.log(`Storing document ${documentName}`);
},
async onDisconnect({ documentName, context }) {
// User disconnected
console.log(`User ${context.user.id} left ${documentName}`);
},
});
server.listen();
Scaling Considerations
┌─────────────────────────────────────────────────────────────────────────────┐
│ SCALING COLLABORATIVE EDITING │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ Challenge 1: WebSocket Stickiness │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ Problem: Clients editing same doc must reach same server │ │
│ │ Solution: Sticky sessions by document ID │ │
│ │ │ │
│ │ # nginx config │ │
│ │ upstream websocket { │ │
│ │ hash $arg_doc consistent; # Route by doc query param │ │
│ │ server ws1:1234; │ │
│ │ server ws2:1234; │ │
│ │ } │ │
│ │ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │
│ Challenge 2: Cross-Server Sync │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ Problem: What if stickiness fails or users are on different DCs? │ │
│ │ Solution: Redis pub/sub for cross-server updates │ │
│ │ │ │
│ │ // Each server subscribes to document channels │ │
│ │ redis.subscribe(`doc:${docId}`, (update) => { │ │
│ │ Y.applyUpdate(doc, update); │ │
│ │ broadcastToLocalClients(update); │ │
│ │ }); │ │
│ │ │ │
│ │ // When receiving update from client, publish to Redis │ │
│ │ redis.publish(`doc:${docId}`, update); │ │
│ │ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │
│ Challenge 3: Memory Management │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ Problem: Documents accumulate tombstones, memory grows │ │
│ │ Solutions: │ │
│ │ 1. Enable Yjs garbage collection (gc: true) │ │
│ │ 2. Periodic snapshots (compact document state) │ │
│ │ 3. Evict inactive documents from memory │ │
│ │ │ │
│ │ // Document eviction │ │
│ │ const EVICT_AFTER = 30 * 60 * 1000; // 30 minutes │ │
│ │ setInterval(() => { │ │
│ │ docs.forEach((doc, name) => { │ │
│ │ if (doc.conns.size === 0 && doc.lastAccess < threshold) { │ │
│ │ persistAndEvict(name, doc); │ │
│ │ } │ │
│ │ }); │ │
│ │ }, 60000); │ │
│ │ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
React Integration: The State Layer
Binding Yjs to React with useSyncExternalStore
// hooks/useYjs.ts
import { useSyncExternalStore, useCallback } from 'react';
import * as Y from 'yjs';
// Generic hook for any Yjs shared type
export function useYjsValue<T>(
yType: Y.Text | Y.Array<any> | Y.Map<any>,
selector: (yType: any) => T
): T {
const subscribe = useCallback(
(callback: () => void) => {
yType.observe(callback);
return () => yType.unobserve(callback);
},
[yType]
);
const getSnapshot = useCallback(() => selector(yType), [yType, selector]);
return useSyncExternalStore(subscribe, getSnapshot, getSnapshot);
}
// Specialized hooks
export function useYText(ytext: Y.Text): string {
return useYjsValue(ytext, (t) => t.toString());
}
export function useYTextDelta(ytext: Y.Text): Y.Delta {
return useYjsValue(ytext, (t) => t.toDelta());
}
export function useYArray<T>(yarray: Y.Array<T>): T[] {
return useYjsValue(yarray, (a) => a.toArray());
}
export function useYMap<T>(ymap: Y.Map<T>): Map<string, T> {
return useYjsValue(ymap, (m) => new Map(m.entries()));
}
// Usage
function DocumentTitle({ ydoc }: { ydoc: Y.Doc }) {
const titleText = ydoc.getText('title');
const title = useYText(titleText);
const handleChange = (e: React.ChangeEvent<HTMLInputElement>) => {
titleText.delete(0, titleText.length);
titleText.insert(0, e.target.value);
};
return <input value={title} onChange={handleChange} />;
}
Provider Pattern for Document Context
// context/DocumentContext.tsx
import { createContext, useContext, useEffect, useState } from 'react';
import * as Y from 'yjs';
import { WebsocketProvider } from 'y-websocket';
import { Awareness } from 'y-protocols/awareness';
interface DocumentContextValue {
doc: Y.Doc;
provider: WebsocketProvider;
awareness: Awareness;
connected: boolean;
synced: boolean;
}
const DocumentContext = createContext<DocumentContextValue | null>(null);
export function useDocument() {
const ctx = useContext(DocumentContext);
if (!ctx) throw new Error('useDocument must be used within DocumentProvider');
return ctx;
}
interface DocumentProviderProps {
documentId: string;
children: React.ReactNode;
}
export function DocumentProvider({ documentId, children }: DocumentProviderProps) {
const [value, setValue] = useState<DocumentContextValue | null>(null);
useEffect(() => {
const doc = new Y.Doc();
const provider = new WebsocketProvider(
process.env.NEXT_PUBLIC_WS_URL!,
documentId,
doc,
{ connect: true }
);
const awareness = provider.awareness;
// Track connection state
let connected = false;
let synced = false;
const updateState = () => {
setValue({
doc,
provider,
awareness,
connected,
synced,
});
};
provider.on('status', ({ status }: { status: string }) => {
connected = status === 'connected';
updateState();
});
provider.on('synced', ({ synced: isSynced }: { synced: boolean }) => {
synced = isSynced;
updateState();
});
updateState();
return () => {
provider.disconnect();
doc.destroy();
};
}, [documentId]);
if (!value) {
return <LoadingSpinner />;
}
return (
<DocumentContext.Provider value={value}>
{children}
</DocumentContext.Provider>
);
}
Block-Based Editor Component
// components/Editor.tsx
import { useDocument } from '@/context/DocumentContext';
import { useYArray, useYMap } from '@/hooks/useYjs';
import { Block } from './Block';
import { RemoteCursors } from './RemoteCursors';
export function Editor() {
const { doc, awareness, synced } = useDocument();
// Subscribe to block order
const blockIds = useYArray(doc.getArray<string>('blocks'));
const blockMap = doc.getMap('blockMap');
if (!synced) {
return <EditorSkeleton />;
}
return (
<div className="editor-container">
<RemoteCursors awareness={awareness} />
{blockIds.map((blockId, index) => (
<Block
key={blockId}
blockId={blockId}
blockMap={blockMap}
awareness={awareness}
index={index}
/>
))}
<AddBlockButton
onAdd={(type) => {
const newBlockId = createBlock(doc, type);
// New block is automatically synced
}}
/>
</div>
);
}
// Individual block component
function Block({
blockId,
blockMap,
awareness,
index,
}: {
blockId: string;
blockMap: Y.Map<Y.Map<any>>;
awareness: Awareness;
index: number;
}) {
const blockData = blockMap.get(blockId);
if (!blockData) return null;
const type = blockData.get('type') as string;
const content = blockData.get('content') as Y.Text;
// Track cursor position
const handleSelectionChange = () => {
const selection = window.getSelection();
if (!selection) return;
// Convert DOM selection to Yjs position
// (This requires editor-specific logic)
const anchor = getYjsPosition(selection.anchorNode, selection.anchorOffset);
const head = getYjsPosition(selection.focusNode, selection.focusOffset);
awareness.setLocalStateField('cursor', {
blockId,
anchor,
head,
});
};
switch (type) {
case 'paragraph':
return (
<ParagraphBlock
content={content}
onSelectionChange={handleSelectionChange}
/>
);
case 'heading':
return <HeadingBlock content={content} />;
case 'code':
return <CodeBlock content={content} />;
default:
return <div>Unknown block type: {type}</div>;
}
}
Integrating with Rich Text Editors
For production use, integrate Yjs with established editors:
// TipTap (ProseMirror-based) - Recommended
import { useEditor, EditorContent } from '@tiptap/react';
import StarterKit from '@tiptap/starter-kit';
import Collaboration from '@tiptap/extension-collaboration';
import CollaborationCursor from '@tiptap/extension-collaboration-cursor';
function TipTapEditor({ ydoc, awareness, user }) {
const editor = useEditor({
extensions: [
StarterKit.configure({ history: false }), // Disable built-in history
Collaboration.configure({
document: ydoc,
field: 'content', // Y.XmlFragment name
}),
CollaborationCursor.configure({
provider: awareness,
user: {
name: user.name,
color: user.color,
},
}),
],
});
return <EditorContent editor={editor} />;
}
// Slate
import { withYjs, YjsEditor, withCursors } from '@slate-yjs/core';
import { createEditor } from 'slate';
function SlateEditor({ ydoc, awareness, user }) {
const [editor] = useState(() => {
const sharedType = ydoc.get('content', Y.XmlText);
const e = withCursors(
withYjs(createEditor(), sharedType),
awareness,
{ data: user }
);
return e;
});
useEffect(() => {
YjsEditor.connect(editor);
return () => YjsEditor.disconnect(editor);
}, [editor]);
return (
<Slate editor={editor}>
<Editable />
</Slate>
);
}
// Lexical
import { LexicalComposer } from '@lexical/react/LexicalComposer';
import { CollaborationPlugin } from '@lexical/react/LexicalCollaborationPlugin';
function LexicalEditor({ documentId, user }) {
return (
<LexicalComposer initialConfig={editorConfig}>
<CollaborationPlugin
id={documentId}
providerFactory={(id, yjsDocMap) => {
const doc = new Y.Doc();
yjsDocMap.set(id, doc);
return new WebsocketProvider(WS_URL, id, doc);
}}
cursorColor={user.color}
username={user.name}
/>
<RichTextPlugin />
</LexicalComposer>
);
}
Handling Edge Cases
Offline Support
// hooks/useOfflineSync.ts
import { IndexeddbPersistence } from 'y-indexeddb';
export function useOfflineSupport(doc: Y.Doc, documentId: string) {
useEffect(() => {
// Persist to IndexedDB for offline access
const indexeddbProvider = new IndexeddbPersistence(documentId, doc);
indexeddbProvider.on('synced', () => {
console.log('Loaded from IndexedDB');
});
return () => {
indexeddbProvider.destroy();
};
}, [doc, documentId]);
}
// Offline indicator
function useOnlineStatus() {
const [online, setOnline] = useState(navigator.onLine);
useEffect(() => {
const handleOnline = () => setOnline(true);
const handleOffline = () => setOnline(false);
window.addEventListener('online', handleOnline);
window.addEventListener('offline', handleOffline);
return () => {
window.removeEventListener('online', handleOnline);
window.removeEventListener('offline', handleOffline);
};
}, []);
return online;
}
// UI feedback
function ConnectionStatus() {
const { connected, synced } = useDocument();
const online = useOnlineStatus();
if (!online) {
return (
<div className="status offline">
Offline - changes will sync when you reconnect
</div>
);
}
if (!connected) {
return <div className="status connecting">Connecting...</div>;
}
if (!synced) {
return <div className="status syncing">Syncing...</div>;
}
return <div className="status connected">All changes saved</div>;
}
Undo/Redo
import { UndoManager } from 'yjs';
function useUndoManager(doc: Y.Doc, trackedOrigins: Set<any> = new Set([null])) {
const [undoManager] = useState(() => {
// Track specific shared types
const content = doc.getText('content');
return new UndoManager(content, {
trackedOrigins,
captureTimeout: 500, // Group changes within 500ms
});
});
const undo = useCallback(() => {
undoManager.undo();
}, [undoManager]);
const redo = useCallback(() => {
undoManager.redo();
}, [undoManager]);
const canUndo = useSyncExternalStore(
(cb) => {
undoManager.on('stack-item-added', cb);
undoManager.on('stack-item-popped', cb);
return () => {
undoManager.off('stack-item-added', cb);
undoManager.off('stack-item-popped', cb);
};
},
() => undoManager.canUndo()
);
const canRedo = useSyncExternalStore(
(cb) => {
undoManager.on('stack-item-added', cb);
undoManager.on('stack-item-popped', cb);
return () => {
undoManager.off('stack-item-added', cb);
undoManager.off('stack-item-popped', cb);
};
},
() => undoManager.canRedo()
);
return { undo, redo, canUndo, canRedo };
}
// Important: Scope undo to local changes only
function createTransaction(doc: Y.Doc, fn: () => void) {
doc.transact(fn, 'local'); // Mark as local origin
}
// Configure UndoManager to only track local changes
const undoManager = new UndoManager(sharedType, {
trackedOrigins: new Set(['local']),
});
Conflict Resolution UI
// For complex conflicts (e.g., block deletion while being edited)
interface Conflict {
type: 'concurrent-delete' | 'concurrent-modify';
blockId: string;
localChange: any;
remoteChange: any;
timestamp: number;
}
function ConflictResolutionDialog({ conflict, onResolve }) {
return (
<Dialog open>
<DialogTitle>Conflict Detected</DialogTitle>
<DialogContent>
<p>
Another user modified this content while you were editing.
How would you like to resolve this?
</p>
<div className="conflict-preview">
<div className="your-version">
<h4>Your version</h4>
<pre>{JSON.stringify(conflict.localChange, null, 2)}</pre>
</div>
<div className="their-version">
<h4>Their version</h4>
<pre>{JSON.stringify(conflict.remoteChange, null, 2)}</pre>
</div>
</div>
</DialogContent>
<DialogActions>
<Button onClick={() => onResolve('keep-mine')}>Keep mine</Button>
<Button onClick={() => onResolve('keep-theirs')}>Keep theirs</Button>
<Button onClick={() => onResolve('keep-both')}>Keep both</Button>
</DialogActions>
</Dialog>
);
}
Performance Optimization
┌─────────────────────────────────────────────────────────────────────────────┐
│ PERFORMANCE OPTIMIZATION STRATEGIES │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ 1. Awareness Throttling │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ // Don't send cursor position on every keystroke │ │
│ │ const throttledUpdateCursor = throttle((pos) => { │ │
│ │ awareness.setLocalStateField('cursor', pos); │ │
│ │ }, 50); // 50ms throttle │ │
│ │ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │
│ 2. Update Batching │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ // Batch rapid changes into single update │ │
│ │ doc.transact(() => { │ │
│ │ ytext.delete(0, 5); │ │
│ │ ytext.insert(0, 'Hello'); │ │
│ │ ytext.format(0, 5, { bold: true }); │ │
│ │ }); // Single update sent │ │
│ │ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │
│ 3. Selective Observation │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ // Only observe visible blocks │ │
│ │ useEffect(() => { │ │
│ │ const blockData = blockMap.get(blockId); │ │
│ │ if (!blockData || !isVisible) return; │ │
│ │ │ │
│ │ const content = blockData.get('content'); │ │
│ │ content.observe(handleUpdate); │ │
│ │ │ │
│ │ return () => content.unobserve(handleUpdate); │ │
│ │ }, [blockId, isVisible]); │ │
│ │ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │
│ 4. Document Snapshots │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ // Periodically snapshot and compact │ │
│ │ async function compactDocument(doc: Y.Doc) { │ │
│ │ const snapshot = Y.encodeStateAsUpdate(doc); │ │
│ │ │ │
│ │ // Create fresh doc from snapshot (removes tombstones) │ │
│ │ const freshDoc = new Y.Doc(); │ │
│ │ Y.applyUpdate(freshDoc, snapshot); │ │
│ │ │ │
│ │ // freshDoc is now a clean copy │ │
│ │ return freshDoc; │ │
│ │ } │ │
│ │ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │
│ 5. Lazy Loading Large Documents │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ // Load blocks on demand │ │
│ │ const { data: blocks } = useInfiniteQuery({ │ │
│ │ queryKey: ['blocks', documentId], │ │
│ │ queryFn: ({ pageParam = 0 }) => │ │
│ │ loadBlockRange(documentId, pageParam, 50), │ │
│ │ getNextPageParam: (lastPage) => lastPage.nextCursor, │ │
│ │ }); │ │
│ │ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
Production Checklist
┌─────────────────────────────────────────────────────────────────────────────┐
│ COLLABORATIVE EDITING PRODUCTION CHECKLIST │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ Core Infrastructure │
│ □ WebSocket server with authentication │
│ □ Document persistence (S3/database) │
│ □ Redis pub/sub for multi-server │
│ □ Sticky sessions or cross-server sync │
│ │
│ Client Implementation │
│ □ Yjs document with appropriate shared types │
│ □ WebSocket provider with reconnection │
│ □ IndexedDB persistence for offline │
│ □ Awareness protocol for presence │
│ □ Undo/redo scoped to local changes │
│ │
│ User Experience │
│ □ Connection status indicator │
│ □ Sync status ("All changes saved") │
│ □ Remote cursor rendering │
│ □ User presence list │
│ □ Offline mode indication │
│ □ Conflict resolution UI (if needed) │
│ │
│ Performance │
│ □ Awareness update throttling │
│ □ Update batching with transactions │
│ □ Selective observation (visible blocks only) │
│ □ Document compaction strategy │
│ □ Memory management for large docs │
│ │
│ Security │
│ □ WebSocket authentication │
│ □ Document-level authorization │
│ □ Rate limiting on updates │
│ □ Input sanitization (XSS prevention) │
│ │
│ Monitoring │
│ □ WebSocket connection metrics │
│ □ Document sync latency │
│ □ Update frequency per document │
│ □ Memory usage per server │
│ □ Error tracking for sync failures │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
The Architecture Summary
┌─────────────────────────────────────────────────────────────────────────────┐
│ COMPLETE ARCHITECTURE OVERVIEW │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ Client (React) │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────────┐ │ │
│ │ │ TipTap/ │ │ Y.Doc │ │ WebsocketProvider │ │ │
│ │ │ Slate/ │◄─┤ │◄─┤ │ │ │
│ │ │ Lexical │ │ - Y.Text │ │ - Connection mgmt │ │ │
│ │ │ │ │ - Y.Array │ │ - Sync protocol │ │ │
│ │ │ (Editor) │ │ - Y.Map │ │ - Awareness │ │ │
│ │ └─────────────┘ └─────────────┘ └───────────┬─────────────┘ │ │
│ │ │ │ │
│ │ ┌─────────────────────────────────────────────┴───────────────┐ │ │
│ │ │ IndexeddbPersistence │ │ │
│ │ │ (Offline support) │ │ │
│ │ └─────────────────────────────────────────────────────────────┘ │ │
│ │ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │ │
│ │ WebSocket │
│ │ │
│ Server (Node.js / Hocuspocus) ▼ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────────┐ │ │
│ │ │ Auth │ │ Y.Doc │ │ Redis Pub/Sub │ │ │
│ │ │ Layer │─►│ (Server) │◄─┤ │◄────┼─► │
│ │ │ │ │ │ │ Cross-server sync │ │ │
│ │ └─────────────┘ └──────┬──────┘ └─────────────────────────┘ │ │
│ │ │ │ │
│ │ ┌───────────────────────┴─────────────────────────────────────┐ │ │
│ │ │ Persistence │ │ │
│ │ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────────┐ │ │ │
│ │ │ │ PostgreSQL │ │ S3 │ │ Backup/ │ │ │ │
│ │ │ │ (Metadata) │ │ (Updates) │ │ Snapshots │ │ │ │
│ │ │ └─────────────┘ └─────────────┘ └─────────────────┘ │ │ │
│ │ └─────────────────────────────────────────────────────────────┘ │ │
│ │ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
The Bottom Line
Building real-time collaborative editing is one of the harder distributed systems problems in frontend engineering. The key decisions:
-
CRDTs over OT for new projects. The math guarantees correctness, libraries like Yjs are battle-tested, and you get offline support for free.
-
Yjs for the CRDT layer. It's the most mature, best-performing, and most widely adopted solution. Don't roll your own.
-
Separate document state from awareness. Document data (CRDTs, persisted) and ephemeral state (cursors, presence) have different requirements.
-
Use established editor integrations. TipTap, Slate, or Lexical with Yjs bindings will save you months of work.
-
Plan for scale from day one. Redis pub/sub, sticky sessions, document eviction—these aren't premature optimizations for collaborative features.
The hard part isn't the algorithm. It's the user experience: showing presence without overwhelming, handling offline gracefully, making conflicts invisible, and keeping the editor fast with 10 simultaneous users typing. That's where the real engineering happens.
What did you think?