Click to play on YouTube.
Mar 5, 2017 Tags: art, programming
This post is a bit of a spiritual successor to one I did about glitch art with RMagick back in 2015.
One of my favorite aesthetics is that of video mosaics. There’s something about firehoses of information and pattern emergence that really appeals to me, and video mosaics embody both of those properties.
I had already
written a script
to produce NxM mosaics using ffmpeg, but it had two obvious shortcomings:
As far as I can tell, ffmpeg’s filter_complex syntax doesn’t support
source-changing, random or in a fixed order. This means that the each tile
in the mosaic can only display one file.
When a source in the mosaic ends, ffmpeg gives only only two options:
end the entire output, or freeze that tile at the last frame and continue
rendering until all sources have ended. While both of these are probably
useful for things like video-camera mosaics and side-by-side source comparisons,
they both result in unnatural looking mosaics that don’t loop well.
To illustrate these shortcomings, here’s a 10x10 mosaic rendered
with filter_complex:
As you can see, the shorter sources simply freeze when they end, drawing your eye to the remaining ones. This looks really bad, and defeats the entire point of the mosaic.
To solve this, I came up with a pipeline:
N seconds long.The first step was pretty simple, since Giphy provides a free beta key
for their API. The end result was
giphy-fetch,
which is invoked like this:
1
$ giphy-fetch --verbose --directory cat_gifs "cats"
yielding 100 GIFs under the cat_gifs directory:
1
2
3
4
5
6
Querying Giphy for 'cats'...
Saving cat_gifs/GvD9nPHWj86VG.gif...
Saving cat_gifs/5r5J4JD9miis.gif...
Saving cat_gifs/l0HlGRDhPTqVEvhCw.gif...
Saving cat_gifs/UotLuplZSzKRa.gif...
[repeat 100 times]
Of course, since each GIF has a different resolution and color palette, they
need to be normalized prior to combination. ImageMagick’s convert kinda
worked for this but was incredibly slow, so I went with
gifsicle instead.
This was accomplished with
another quick script (in bash this time),
normalize-gifs:
1
$ DIMS=500x500 normalize-gifs cat_gifs/*.gif
yielding a directory with the same resolution (500x500) containing
the normalized GIFs:
Grouping the GIFs into random collections with the same total duration
was slightly trickier, but made possible by exiftool’s -Duration flag:
1
$ exiftool -quiet -Duration normalized_gifs/*.gif | awk '{ print $3 }' | sed '/^$/d' | sed '$d'
yielding:
1
2
3
4
5
6
7
8
9
3.78
5.07
3.60
4.05
0.40
4.20
7.07
1.68
[...]
These then need to be mapped to their original sources, and used to build
groups whose durations are each N seconds (plus or minus 1 second, since
we don’t have perfectly sized GIFs to pad everything out). Once grouped,
gifsicle can be used again to merge each individual group into one big GIF:
1
$ gifsicle -w --colors 256 ${group} > groups/group${n}.gif
This was all Ruby-ified and wrapped in a script,
gif-length-groups,
which gets invoked like this:
1
2
# build 100 groups, each 60 seconds long, with no GIF longer than 5s
$ gif-length-groups -n 100 -l 60 -i 5 500x500/*.gif
yielding the groups directory:
each of which looks something like this:
Now we’re finally ready to use
ffmpeg-mosaic:
Click to play on YouTube.
1
2
# if you rendered less than NxM groups, you can add -r to shuf to repeat them
$ ffmpeg-mosaic 1920x1080 10x10 mosaic.mp4 $(shuf -e -n 100 300x300/*.gif)
yielding the final product:
Of course, we can do less dense layouts as well:
These all flow smoothly from one GIF to another, bypassing the limitations
of filter_complex.
My original plan was to use these mosaics as a screensaver via XScreenSaver + mpv, although I quickly realized that XScreenSaver doesn’t have an easy way to orient one of the outputs to fit my vertical monitor. Still, it looks pretty awesome on my horizontal monitors:
A friend suggesting playing the mosaic on an old CRT television, which I think would also look pretty awesome (especially with some static or interference introduced). I’ll post an update if I can get my hands on the right equipment for that.
You can find the scripts used in this post on my site, on the
snippets page. You can also check out
my YouTube channel, where I’ll post more
mosaics as I continue to experiment. Feel free to download the generated mosaics from
there using youtube-dl.
As it turns out, generating lots of unique large GIFs takes up a decent amount of space, enough to make it (currently impractical for me to distribute the normalized GIFs and GIF groups that I rendered.
If you’d really like to use the data that I used (instead of just pulling from Giphy or another source), send me an email and we can figure out the best way to do a point-to-point transfer. Otherwise, check out “psychedelic”, “hacker”, “cyberpunk”, and “vaporwave” via the Giphy API - most of the GIFs I used came from those queries.
Happy hacking!
- William
Click to play on YouTube.
Click to play on YouTube.
Click to play on YouTube.