Inside the Protocol: Real-Time Terminal Streaming from Mac to iPhone

Terminal streaming is the core of Tactic Remote. Every feature — approvals, notifications, session monitoring — depends on the iPhone displaying an accurate, responsive representation of what's happening in your Mac's terminal. Getting this right required solving several non-obvious problems at the protocol level.

This post walks through how our streaming protocol works, from tmux capture to iPhone rendering, and the engineering tradeoffs we made along the way.

The Pipeline

Terminal data flows through five stages:

1. tmux capture — The Mac companion app reads terminal content from tmux panes.
2. Diff computation — We compare the current capture against the last transmitted state.
3. Serialization — Changed regions are packed into a compact binary frame.
4. WebSocket transport — Frames are sent to the iPhone over a persistent WebSocket connection.
5. ANSI rendering — The iPhone client interprets escape sequences and renders styled text.

Each stage has its own latency budget. Our target is end-to-end delivery in under 100ms on a local network, leaving meaningful headroom within each stage.

tmux Pane Capture

Tactic Remote manages Claude Code sessions through tmux. This was an early architectural decision that pays dividends for streaming: tmux maintains a virtual terminal buffer that we can read programmatically without intercepting the actual PTY stream.

We use tmux capture-pane with the -p and -e flags to extract pane content with ANSI escape sequences preserved. Captures run on a 50ms polling interval during active streaming — fast enough to feel real-time, infrequent enough to avoid meaningful CPU overhead.

One subtlety: capture-pane returns the visible pane content plus scrollback. We capture only the visible viewport by default and stream scrollback on demand when the user scrolls up on iPhone. This keeps the per-frame payload small for the common case.

The capture process adds approximately 3-8ms per frame on a modern Mac, measured across M1 through M4 hardware.

Differential Updates

Sending the full terminal buffer on every frame would be wasteful. A typical 80x24 terminal pane contains roughly 1,920 characters. With ANSI sequences, a full frame can reach 8-12 KB. At 20 frames per second, that's 160-240 KB/s of bandwidth — acceptable on local Wi-Fi but problematic for cellular connections through Cloudflare Tunnel.

Instead, we compute line-level diffs between consecutive frames. Most terminal updates affect only a few lines — a new log line appears, a progress bar advances, a cursor moves. Our diff algorithm identifies changed lines by hash comparison and transmits only the modified content.

In practice, differential updates reduce average frame size by 85-92%. A typical diff frame during active Claude Code output is 400-800 bytes. During idle periods (waiting for user input or API responses), no frames are sent at all — we only transmit when content changes.

The Wire Format

Diff frames are serialized into a compact binary format rather than JSON. Each frame contains:

• A 4-byte header: protocol version (1 byte), frame type (1 byte), and payload length (2 bytes).
• A sequence number for ordering and loss detection.
• An array of line update records, each containing the line index, the new content (UTF-8 with embedded ANSI sequences), and a CRC32 checksum for integrity verification.

We chose a custom binary format over Protocol Buffers or MessagePack after benchmarking showed that the serialization overhead for small, frequent messages was dominated by library initialization costs. Our hand-rolled serializer adds under 1ms per frame.

WebSocket Transport

The streaming connection uses a single persistent WebSocket. We evaluated Server-Sent Events (SSE) and raw TCP, but WebSocket offered the best combination of bidirectional communication (needed for scroll requests and flow control), broad network compatibility, and built-in framing.

Connection management handles three scenarios:

Local network — The iPhone connects directly to the Mac companion's local server. Round-trip time is typically 2-8ms. Total streaming latency (capture to render) averages 40-60ms.

Cloudflare Tunnel — Traffic routes through Cloudflare's edge network. Round-trip time increases to 30-80ms depending on geography. Total streaming latency averages 120-200ms. Still responsive enough for monitoring, though users notice the difference during fast-scrolling output.

Network transitions — When the iPhone switches between Wi-Fi and cellular (or between networks), the WebSocket connection drops. Our reconnection logic re-establishes the connection within 1-2 seconds and requests a full-frame resync to restore state. During reconnection, a visual indicator on the iPhone shows connection status so users know why output has paused.

ANSI Escape Sequence Handling

Terminal output from Claude Code is rich with ANSI escape sequences — colors for syntax highlighting, bold for emphasis, cursor positioning for progress indicators. Preserving these sequences accurately on iPhone is essential for readability.

Our iOS renderer handles the most common SGR (Select Graphic Rendition) sequences: 16 standard colors, 256-color palette, 24-bit true color, bold, italic, underline, and inverse. We intentionally do not support cursor movement sequences (CSI H, CSI A/B/C/D) in the rendered view — these are consumed during tmux capture and reflected in line positioning rather than transmitted as raw escape codes.

One challenge specific to mobile: dark mode. Terminal color schemes designed for dark backgrounds render poorly when iOS is in light mode (and vice versa). We detect the iPhone's current appearance mode and apply a color mapping that preserves contrast ratios. This is not a theme — it's a luminance adjustment that keeps the original color intent while ensuring readability.

Performance Budget

We track streaming performance across three percentiles:

These numbers come from production telemetry across our beta user base. The P50 local latency of 15ms means that for most interactions, the iPhone display updates faster than the human eye can perceive the delay.

What We Learned

Building a real-time streaming protocol for terminal output taught us several lessons:

Polling beats event-driven for this use case. We initially tried hooking into tmux's internal update events but found that 50ms polling was simpler, more reliable, and had negligible performance cost.

Line-level diffs are good enough. We prototyped character-level diffing but the additional complexity yielded only a 5-8% further reduction in bandwidth, not worth the maintenance burden.

Binary framing matters at high frequency. For messages sent 20 times per second, every byte of overhead in the serialization format is amplified. JSON's verbosity was measurably costly.

Reconnection UX is as important as connection performance. Users tolerate a 2-second reconnection if they understand what's happening. They perceive a 500ms reconnection as broken if there's no visual feedback.

For implementation details, see our Architecture documentation. If you're building something similar, we're happy to discuss our approach — reach out through the Support page.