TL; DR
- You need more keyframes.
- At the cost of increasing video size.
Introduction
Scrollable videos are normal muted videos that play forwards and backwards as user scrolling the page. Complex animations, 3D transitions for example, are usually hard to implement with pure JavaScript but relatively easy with video creation softwares and dedicated 3D softwares like Adobe AE and Blender. As a solution, scrollable videos could be applied to create complex while interactive animations in your web page.
Apple has many good examples in their product pages. For example their page for Airpods 2.
Implementation
The script to implement this effect is relatively straightforward:
let video = document.getElementById("video");
function play() {
let time = (window.scrollY / window.scrollMaxY) * video.duration;
video.currentTime = time;
requestAnimationFrame(play);
}
play();
and the HTML:
<video id="video" src="<SOURCE>" style="" />
The Problem
If you try to use a normal video, the playback would be extremely choppy.
For example, the following video comes from MDN doc for HTML5 video element:
Simply put, this is intended behavior, as the video is not designed to be played in this way.
Keyframes
mp4 and webm videos are designed to be played in a linear fashion.
In order to save space, the video is encoded with keyframes. The frames in between keyframes are encoded as the difference between the keyframes. When you seek to a point in the video, the player must decode the keyframe and all the frames in between to get to the point you want to watch.
In order to make the video scroll smoothly, you need to add more keyframes (which will increase the size of the video).
With ffmpeg, you can achieve this with the following command:
ffmpeg -i <INPUT> -c:v libx264 -x264-params keyint=15 <OUTPUT>
In which keyint is the number of frames between keyframes. The lower the number, the more keyframes you will have.
For example, with keyint=1, the above example now scrolls smoothly: