Pocket Glass — AR Glass Through Your Phone

Project P15 · Three.js From Zero · Single-Page Build

Open this page on your phone. Tap once for camera + motion. The back-camera becomes the background of a tiny 3D scene — and in the middle of it a single glass cube floats, refracting whatever your camera is pointed at. Tilt the phone and you orbit the cube; the cube picks up the colour and brightness of your room as its environment light. Built in 600 lines of vanilla Three.js. No WebXR, no library, no app.

Best on iPhone or Android (rear camera + gyro). Desktop works too — mouse orbits the cube; the front camera fills the background.

What you're actually building

Five moving parts. None of them are complicated on their own — the magic is the combination:

• A video texture made from getUserMedia({ facingMode: 'environment' }). It becomes scene.background and scene.environment at the same time, so the glass refracts AND reflects the live camera feed.
• One glass cube — a RoundedBoxGeometry wearing a MeshPhysicalMaterial with transmission: 1, thickness: 1.4, ior: 1.5, dispersion: 0.4, clearcoat: 1. Five lines for proper glass.
• A CubeCamera at the cube's centre, low-res (256) but updated every frame. The render target becomes the cube's envMap so the reflections track your actual surroundings as you move the phone.
• The gyro driver — deviceorientation alpha/beta/gamma → spherical-coord camera orbit, smoothed with a One-Euro filter. DeviceOrientationEvent.requestPermission() for iOS 13+, which must be invoked from a user gesture.
• One tap to start everything — iOS requires camera permission AND motion permission BOTH triggered from the same user gesture. Combining them is the difference between "it works" and "it stays at the loading screen forever".

The combined-permission trick

The whole reason this needs a "Tap to enable" button is iOS Safari. You can't ask for the camera OR for motion permission programmatically. Both must originate from a user gesture (a synchronous click handler). And on iOS 13.4+ specifically, motion requires its own permission prompt:

async function startEverything() {
  // 1. Camera FIRST — failure here is the loudest UX problem
  const stream = await navigator.mediaDevices.getUserMedia({
    video: {
      facingMode: { ideal: 'environment' },  // rear cam if available
      width:  { ideal: 1280 },
      height: { ideal: 720  },
    },
    audio: false,
  });
  videoEl.srcObject = stream;
  await videoEl.play();

  // 2. Motion — only iOS Safari 13+ has requestPermission. Others get
  //    DeviceOrientation events for free as soon as you listen.
  if (typeof DeviceOrientationEvent.requestPermission === 'function') {
    try {
      const result = await DeviceOrientationEvent.requestPermission();
      if (result === 'granted') {
        window.addEventListener('deviceorientation', onOrientation);
      }
    } catch {}
  } else {
    window.addEventListener('deviceorientation', onOrientation);
  }
}

Both the getUserMedia call AND the requestPermission call must be reached synchronously from the click handler — no setTimeout, no requestAnimationFrame, no awaiting unrelated work first. Browsers consider any of those to "break" the user gesture and silently swallow the request.

The glass material recipe

const mat = new THREE.MeshPhysicalMaterial({
  color: 0xffffff,
  roughness: 0.02,                  // almost mirror-smooth
  metalness: 0,                     // glass is dielectric, never metal
  transmission: 1.0,                // 100% see-through
  thickness: 1.4,                   // volume — drives refraction strength
  ior: 1.5,                         // real glass: 1.50–1.52
  attenuationDistance: 2.0,          // how far light travels before absorbing
  attenuationColor: 0xeaf6ff,       // faint cool tint as light passes through
  clearcoat: 1.0,                   // outer lacquer reflection
  clearcoatRoughness: 0.0,
  dispersion: 0.4,                  // chromatic prism — splits white edges into RGB
  envMapIntensity: 1.5,
});

dispersion is the one that sells it. Three.js added it in r163 — it's a wavelength-dependent IOR shift that gives glass edges that faint prism-rainbow you see in real life. Without it your "glass" looks like plastic.

Live environment reflection — CubeCamera at low cost

Static envMaps make glass look fake on a moving camera. A CubeCamera renders the scene to a 6-face cube target each frame and feeds it to the material as envMap. At 256² per face with only a background plane visible (we hide the glass during the cube render to avoid recursion), it's cheap enough for phones.

const cubeRT = new THREE.WebGLCubeRenderTarget(256, {
  generateMipmaps: true,
  minFilter: THREE.LinearMipmapLinearFilter,
});
const cubeCam = new THREE.CubeCamera(0.1, 100, cubeRT);
scene.add(cubeCam);
glassMaterial.envMap = cubeRT.texture;

// Each frame:
glass.visible = false;           // avoid sampling itself
cubeCam.position.copy(glass.position);
cubeCam.update(renderer, scene);
glass.visible = true;
renderer.render(scene, camera);

Gyro → camera orbit

deviceorientation gives you three Euler angles — alpha (compass / Z rotation), beta (front-back tilt, X rotation), gamma (left-right tilt, Y rotation). Map gamma to azimuth and beta − 60 to elevation (60° because "neutral phone in hand" is tilted that much forward), smooth with one-Euro, and feed into setFromSphericalCoords:

function onOrientation(e) {
  const gamma = (e.gamma ?? 0);
  const beta  = (e.beta  ?? 0);
  rawAzimuth   = THREE.MathUtils.degToRad(-gamma);
  rawElevation = THREE.MathUtils.degToRad(beta - 60);
}

// Each frame:
azimuth   += (rawAzimuth   - azimuth)   * 0.18;
elevation += (rawElevation - elevation) * 0.18;
const phi = THREE.MathUtils.clamp(Math.PI / 2 - elevation, 0.2, Math.PI - 0.2);
camera.position.setFromSphericalCoords(distance, phi, azimuth);
camera.lookAt(0, 0, 0);

What's worth experimenting with

1. Sphere instead of cube. Refraction inside a sphere does the lens-magnifier thing — fish-eye view of whatever the camera sees behind.
2. A prism. ConeGeometry(1, 1.5, 3) + dispersion 0.6 = a triangular prism that splits whites into rainbows along its edges.
3. Audio-reactive size. Wire a MediaStreamSource → AnalyserNode → cube.scale. The cube pulses with ambient sound.
4. Touch to add cubes. Tap the screen → spawn another glass cube at the tap point projected into the cube's plane.
5. Save the moment. Add a tiny shutter button → renderer.domElement.toBlob → download PNG. Free AR photo mode.

What's next

The same camera-as-background pattern transfers to anything that needs "real world behind 3D content" without WebXR overhead:

• An AR product viewer that uses your room as the env map
• A virtual try-on overlay anchored to face-tracker output
• A "magic mirror" that warps your reflection in real time
• A passthrough scene for visionOS-style demos in plain Safari