CSS GridとWebVTTで再構築された放送の「スクイーズバック」効果

Confession: I did not care much about football as recently as May. But by the time España lifted the trophy last month, I was watching matches I had no stake in, still unable to explain offside. Somewhere in one of those matches, I noticed what the picture on the screen sometimes did: the grassy pitch slides into one corner and scales down, and the space around it opens up and fills with a sponsor card, a second angle, or the studio desk. You don’t miss a moment of the game. I'd seen that effect a bajillion times without really looking too hard at it. It's just what television does. But during one of my infamous 2am can’t-sleep iPhone research sessions, I learned it actually has a name: a squeezeback. The closest thing I’ve seen for this effect on the web is a YouTube video where somebody squeezes the frame in After Effects and renders it out. The layout is a picture, baked in before upload, and the most interactive it gets is a hotspot on top of a motion graphic someone already designed. Which is strange… because a squeeze is just a layout change, and browsers are very good at layout changes! Do it in CSS and the space that opens is a cell you can put a real element in, chosen at playback instead of in post. Link The flexibility of a smooshy CSS grid What happens if we put the player in the center cell of a 3×3 grid where the six outer tracks are collapsed to nothing, then animate the track sizes? A grid with different layouts CheckCopiedCopyCopyCheckCopiedCopyCopy.stage { display: grid; grid-template-columns: 0fr 100fr 0fr; grid-template-rows: 0fr 100fr 0fr; overflow: hidden; transition: grid-template-columns 850ms cubic-bezier(0.65, 0, 0.35, 1), grid-template-rows 850ms cubic-bezier(0.65, 0, 0.35, 1); } .stage[data-layout='right-rail'] { grid-template-columns: 0fr 62fr 38fr; } .stage[data-layout='lower-third'] { grid-template-rows: 0fr 74fr 26fr; } .stage[data-layout='squeeze'] { grid-template-columns: 5fr 63fr 32fr; grid-template-rows: 0fr 82fr 18fr; row-gap: 8.9%; } Woah. That's an entire motion system! It doesn’t even need a transform on the video or scaling wrapper or requestAnimationFrame loop measuring anything. The video is a normal grid item in a cell that's getting smaller, and the panels parked in the other cells get revealed as their track values leave zero. The reveal and the shrink are the same event. The panel was always sitting in that cell at full size. The shrink is just the moment the cell stops hiding it. Isn't CSS neat? That row-gap: 8.9% is the only odd number in there. Percentage gaps resolve against height, and at 16/9, 8.9% of height is 5% of width, which is exactly the 5fr left column. The top track is 0fr, so that one value makes the space above and below at the same time. Equal inset on three sides. Unfortunately, you can't write it in cqw and skip the math since the stage is the query container, and a container can't query itself. Link The timeline is a text file Video.js v10 is a React video player component library: createPlayer, hooks, composable primitives, etc. but what it doesn't do is invent a scheduling system, because the browser already has a perfectly good one: WebVTT. cta-cues.vtt CheckCopiedCopyCopyCheckCopiedCopyCopyWEBVTT away-kit 00:00:04.000 --> 00:00:12.000 right-rail champions-bundle 00:00:26.000 --> 00:00:34.000 squeeze Every cue has an optional identifier line right above the timestamps that isn’t really used very often, but it's perfect for this use case. The identifier becomes the product key, the payload becomes the layout, and the copy and pricing stay in your app keyed by that id. If you’re working on shoppable video, a merchandiser could retime the whole experience by editing a text file, and your pipeline can generate one per asset without a deployment. You can add it to the player as a <track> element and let the browser tell you when something is active: Video player with cues attached CheckCopiedCopyCopyCheckCopiedCopyCopy<MuxVideo src={src} autoPlay muted playsInline loop crossOrigin="anonymous"> <track kind="metadata" label="cta" src="/cta-cues.vtt" default /> </MuxVideo> Video.js v10’s usePlayer takes a selector arg, so selectTextTrack subscribes you to just the text track slice of the store and nothing else. That matters because you aren't re-rendering on every timeupdate, and you get told when the track has actually registered, because tracks come and go while the engine attaches. Use the cues CheckCopiedCopyCopyCheckCopiedCopyCopyimport { usePlayer, selectTextTrack } from '@videojs/react'; const { textTrackList } = usePlayer(selectTextTrack); const ready = textTrackList.some((t) => t.label === 'cta'); The store models tracks as plain descriptors, so once it says yours exists, you can check the live TextTrack to access the cues themselves: Handle the cuechange event CheckCopiedCopyCopyCheckCopiedCopyCopytrack.mode = 'hidden'; track.addEventListener('cuechange', () => { const cue = track.activeCues?.[0]; setActiveCue(cue ? { id: cue.id, layout: cue.text.trim() } : null); }); A text track has three modes: showing paints cues on screen as captions, disabled stops cuechange firing at all, and hidden parses the cues and fires the events without rendering anything - so that’s the one we’re using to fire layout changes. Once you're listening for cuechange instead of polling currentTime on a timer, scrubbing backwards through a cue window, looping and seeking all behave correctly without you writing a line for any of them. You can then bind it to the DOM with one data attribute: <div className="stage" data-layout={cue?.layout ?? 'full'}>One warning: for now, you should build the cues in a file rather than in JavaScript, at least as of writing this post. hls.js clears the cues off every text track when it attaches, and v10 ships a mixin that repairs the damage by finding the <track> element and reloading it. A track you created with addTextTrack() has no element to reload, so it silently stays empty while everything else looks correct. We should make that louder or fix it, but file-based VTT will work for now. Link Let the video light up the room I’ve always liked the gradient effect that my hue lights spill out behind my TV screen, matching the colors off of the display. Let's create that here, too: Glowing gradient canvas CheckCopiedCopyCopyCheckCopiedCopyCopy<canvas ref={canvasRef} width={32} height={18} className="stage__ambient" /> const ctx = canvas.getContext('2d', { willReadFrequently: true }); let lastDraw = 0; const tick = (now) => { frame = requestAnimationFrame(tick); if (now - lastDraw < 100) return; // ten times a second is plenty lastDraw = now; if (!video.videoWidth) return; // nothing decoded yet ctx.drawImage(video, 0, 0, 32, 18); }; frame = requestAnimationFrame(tick); We can use CSS to stretch a canvas across the stage and blur it into mush, so the revealed space gets lit by whatever is on screen. It’s a pretty performant solution too, so you don’t have to worry too much about the cost of implementing this effect. Twinsies colors willReadFrequently: true warns the browser you plan to read this canvas back, and without it the canvas lives on the GPU where getImageData stalls every call. Also, videoRef has to point at the actual <video> element. The Media object you get back from useMedia() is a runtime-agnostic wrapper, and drawImage wants a CanvasImageSource, so handing it the wrapper leaves you with a blank canvas and nothing in the console to explain why. We can even read and use the frame colors and pull an accent color for the buy button background color. My first attempt at this averaged the red, green, and blue channels and produced the same olive brown on every frame, because the bright sky and the dark shadows from the video cancel out and land right in the middle of the color wheel. The fix was to treat the frame’s hue as an angle instead of just an averaged number: Sample colors from an angle CheckCopiedCopyCopyCheckCopiedCopyCopyconst weight = saturation * (1 - Math.abs(