[Previous] Elliot's Thoughts on Pricing | Home | [Next] Mistaken Government Violence

Video Troubleshooting with Keyframes

I made a 65 minute screencast with Screenflow and knew I had a problem. I normally record in a second space (with the spaces feature of macOS) that doesn't have anything private in it. If I ever go back to my regular space then private information gets recorded, like my Messages window, and I have to edit it out.

I paused and checked my messages. Then I accidentally unpaused with my messages showing. I paused it again immediately and hit my marker hotkey to tell Screenflow to mark that timestamp.

After recording I saw Screenflow did not mark the timestamp. After some testing, I discovered Screenflow doesn't create markers while paused. That's dumb.

So I had a half second problem in a 65 minute video. I didn't have much memory of where in the video the problem was. I tried to find the spot in Screenflow by dragging the scrubber around quickly. Then I exported the video and tried watching it at high speed with VLC and quicktime. VLC didn't work well, at 16x or higher playback it often gets stuck on one frame and stops showing anything. Quicktime can play at 60x without messing up, and I tried it, but that was too fast for me to see the problem. I tried 30x too and checked a couple spots where I saw something flash, but they were just other relatively large changes on the screen.

I was unwilling to sit there and watch the whole video at more like 4x – and pay close attention to spot a fraction-of-a-second problem (it'd only show up for maybe an eighth of a second at 4x).

So I googled for how to find keyframes. I figured there'd be a lot of motion when I instantly switch from the screencast to my messages window, and a keyframe would get created.

First I found this:

http://superuser.com/questions/885452/extracting-the-index-of-key-frames-from-a-video-using-ffmpeg

time ffprobe -select_streams v -show_frames -show_entries frame=pict_type -of csv myvideo.mp4 | grep -n I | cut -d ':' -f 1 > frame_indices.txt

It takes 10 minutes to run and a bunch of CPU but it worked. While it ran I found this:

https://gpac.wp.mines-telecom.fr/2015/11/02/check-key-frame-alignment-with-mp4box/

time MP4Box -std -diso myvideo.mp4 2>&1 | grep SyncSampleEntry > keyframes.txt

After a quick "brew install mp4box", that ran in a quarter second.

I saw keyframes like this:

1 26 51 76 101 126 151 176 201 226 251 276 301 326 351

I skimmed around and it kept going like that. A keyframe every 25 frames. Uh oh. That's not very useful. It did give me the idea that I could get my video sizes a lot smaller if I could generate fewer keyframes. When I'm writing I don't need many keyframes.

The end of the list looked liked this:

117293 117318 117343 117368 117393 117418 117443 117468 117493 117518

That's still every 25 frames. Same boring pattern.

But wait. The pattern shifted. It starts at 18 instead of 1. Something must have changed somewhere.

I manually performed binary search to find the change point. How? Go to the middle. If I see a frame ending with 18, I'm still too near the end, go up a bunch. If I see a frame ending with 1, I'm too near the start, go down a bunch. After I narrowed it down a few times that way, then I scrolled quickly in the right direction until I saw the pattern change.

36201 36226 36251 36276 36301 36319 36343 36368 36393 36418 36443 36468 36493

At that point, scrolling down, the last digits weren't just 1 and 6 every time, there's different digits.

36319 frames, at 30 frames per second, is 1211 seconds. That's 20 minutes and 11 seconds.

I checked the video and that was my problem. Right at 20:11 I showed my chat messages for half a second. It worked perfectly. I edited it out, problem solved!


Elliot Temple on October 2, 2016

Comments (2)

Here's the command I've added to my .profile:

function keyframes { MP4Box -std -diso "$1" 2>&1 | grep SyncSampleEntry | egrep -o "\d+" > ${2:-keyframes}.txt;}

curi at 4:51 PM on October 2, 2016 | #6716
this is pretty interesting

http://www.lighterra.com/papers/videoencodingh264/

> x264's default is 250 frames (8.3 seconds at 30fps), but for no good reason the x264Encoder QuickTime plug-in's default is only 60 frames (2 seconds). The x264 team have recently been using 500 frames (16.7 seconds) for their settings at the annual MPEG-4 AVC/H.264 Video Codecs Comparison competition, but that's definitely pushing things a bit too far. Digital TV normally uses 1 or 2 seconds, but that's deliberately quite short to make channel switching fast and to have quick error recovery in case of interference. DVD uses a very short ~0.5 second keyframe interval, and Blu-ray 1 second, because they use very high bitrates (so quality isn't an issue) and they want to guarantee good fast-forwarding behavior despite being read from a relatively slow optical disc. The most common recommendation for Internet video is 10 seconds.

Anonymous at 5:46 PM on October 2, 2016 | #6719

What do you think?

(This is a free speech zone!)