3D Depth Maps & WebGL Displacement

Scrollsequence features built-in support for monocular 3D depth maps. By pairing each color frame with a corresponding grayscale depth image, Scrollsequence creates real-time 2.5D parallax, mouse tilt, and camera displacement effects using GPU-accelerated WebGL shaders.

How Depth Displacement Works

Grayscale Encoding:
- In each depth map frame, pixel brightness represents spatial depth from the camera lens.
- White (#ffffff): Represents elements closest to the foreground.
- Black (#000000): Represents elements furthest in the background.
Real-Time Shader Displacement:
- The WebGL rendering engine offsets pixels along the X and Y axes in proportion to their depth value and user interaction (e.g. mouse cursor movements, gyroscope device orientation, or scroll acceleration).
- Foreground objects shift more than background objects, producing a lifelike 3D stereoscopic illusion.

Shared Depth Sequences

Because depth maps represent low-frequency geometry rather than high-frequency color detail, depth maps can be rendered at lower resolutions (e.g., 720px wide) and shared across all color tiers (1280, 1920, etc.).

In the manifest:


"files": {
  "i": "1920/%04d.jpg",
  "d": "depthmaps/%04d.jpg"
}

This drastically reduces total file size while preserving high visual fidelity.

Enabling Depth in Your Sequence

1. Generating Depth via CLI

Run vid2ssq with the --depth flag:


npx scrollsequence vid2ssq input.mp4 --name 3d-hero --depth

2. Enabling Tilt in React

Pass depthMapsTilt or interactive tilt properties to <Canvas2d>:


<Canvas2d
  depthMapsTilt={0.28}
  mediaGroups={[/* ... */]}
/>