I wanted to document an experiment I undertook trying out AV1 with Grain Synthesis on particularly grainy footage. I'll go over my thoughts and explore Plex compatibility with this feature.
This is going to be another one of my rare highly-detailed technical posts, so feel free to skip to the conclusion if you'd like.
What kicked this off?
I've been ripping Blu-rays and transcoding them for my Plex server for about 7 years now. In that time I have tweaked my settings a little, but I've broadly settled on h265 as my codec of choice. This is because it is effective at preserving fine details and film grain while still giving you a pretty good compression ratio.
Here's my typical configuration:
- Use handbrake as my encoding application (what can I say, I like GUIs)
- Use the x265 (CPU h265) encoder. Why not use NVENC? It's efficiency and detail preservation sucks for HQ movies.
- Play with the quality (RF) slider and do previews until I am happy I see no artifacting.
- Typically results in 8-12Mbps video.
- Typically that means low grain movies are 16-18 RF, very grainy movies at up to 22 RF.
If you know anything about film grain, it's that it is absolutely hell on compression algorithms. It's very hard to compress grainy footage without making it look bad. Sometimes, you come across a piece of media that is especially difficult, and that's how I found myself exploring grain synthesis this time.
What piece of media happened to kick this off? Columbo Season 9 Episode 2. No spoilers, I haven't sat down to watch it yet.
Why this in particular? I was making my way through encoding my collection of Columbo Blu-rays and noticed that Season 9 in particular was incredibly grainy. Combine that with the slight sharpening filter I tend to add to my encodes (like a bit of salt, to taste) and the final size of my encodes, instead of being around 30-50% of the original size on disc it was almost 90% at around 20Mbps!
So I set out to try this grain synthesis I had heard about.
What is Grain Synthesis?
Grain Synthesis is a novel approach introduced with the AV1 video codec, but which has only really started seeing limited use in the last few years. See this blog post by Netflix for a great explainer.
The TLDR; is that Grain Synthesis is where you:
- Analyze the characteristics of film grain in a particular piece of media
- Denoise the video
- Add the film grain characteristics as metadata in the video file / stream
- When the end user plays back the video, the grain is reapplied as a filter on top of the video
Why do this? To save bandwidth and storage space.
The theory is that by denoising, you make the video easier to compress, while still being able to evoke the feel of film grain convincingly.
Does this work in practice? Read on to find out. (Yes, saying that did make me cringe).
The experiment
The experiment was simple. Target my usual 8-10mbps with AV1 grain synthesis and see how that compares to just crushing the quality down with x265.
Application and encoder choice
After a bit of research, I decided to opt for StaxRip as the application to do my encoding. It supports a lot of AV1 encoders, including:
- rav1e - a Rust-based community-written encoder
- AOMEnc - The reference encoder.
- SVT-AV1 - Intel / Netflix's open source encoder (and it's variants).
Which encoder to use?
Well, when I tried AOMEnc and found it SUPER slow. I encode on a 12 core, 24 thread Ryzen CPU, and got 4fps. There are likely ways to get it to run faster, but I didn't play around too much. rav1e does not seem to support Grain Synthesis right now.
So really that left SVT-AV1. That's the one Handbrake includes, but in the past I found the vanilla SVT-AV1 found on there to give me really blotchy results in dark scenes, so I picked SVT-AV1-PSYEX.
Key settings
While it is technically possible to assemble the grain analysis, denoising and synthesis yourself, the easiest way to use it is actually to use the following two parameters:
--enable-dnl-denoising=1
This turns on denoising. Without this I think grain synthesis is simply added on top of your existing video.
--film-grain=<some integer value>
This sets the strength of the film grain added and, with enable-dnl-denoising=1, the strength of the denoising. Online, people recommend using values of 8-14 depending on the amount of grain.
How I dialled in the settings
I found that the best way to dial-in AV1 grain synthesis was to start with a very high quality setting, then gradually increase the noise reduction through the --film-grain value until I got most of the grain off. I could see this by using a video player that doesn't support / has grain synthesis turned off (more on this in Compatibility).
I would however avoid going too far with the noise reduction strength. No need to remove all the grain. Pushing it too far results in the video into a smeary mess. After all, no denoising algorithm is perfect, and the more you crank it, the more you lose real detail.
Then, once I was happy with the clarity, I'd dial down the quality until I either hit my size target or my acceptable lowest quality.
As for the h265 encode, I apply a Light sharpening filter in Handbrake, then reduce the quality until I hit my target.
Note here that the process with h265 has less levers to pull, so it takes less time to dial it in.
Results
See for yourself! Here's a MEGA link to the final encodes . I make no promise that I will keep that link up long-term, but I will personally hold onto the files so if the link dies in the future, try DMing me and hopefully I'll see it and send them to you.
There are 3 files:
no denoise.mkv- a file encoded with no grain synthesis and very high quality setting to act as a baselineAV1 q25 denoise38.mkv- The grain synthesis file. Quality of 25 with film grain setting of 38.x265 crf22.5.mkv- The h265 file. Used acrfof 22.5. Picked to match the AV1 file size.
I highly recommend using a tool like video-compare to play them side by side. Grain behaviour is not easily comparable in screenshots and should be compared in motion. The command you want is:
video-compare "AV1 q25 denoise38.mkv" "x265 crf22.5.mkv"
Analysis
I call this section "Analysis", but it's quite subjective.
Grain
To me the AV1 file has cleaner grain. It's fairly convincing in motion, if a bit more regular than is realistic. Reminds me a little of some modern shows that probably have added grain in post-production.
By comparison, the h265 version has made the grain is kind of start-stop and affected by motion in the scene. It has necessarily been compressed and looks worst for it. It's not that bad though.
Quality retention
The better quality retention goes to h265 here. When you look at the details around the eyebrows of the male character here, there is perceivably a little more detail preserved. Good looking grain can give an increased perception of sharpness, which makes up some of the ground on the AV1 side.
Compatibility
Here's a very important point we've only touched on so far: What devices actually support AV1 and grain synthesis, and if they don't support it, what happens?
Well, the answer is that when AV1 is supported, but grain synthesis is not (or it's otherwise broken), the grain synthesis tends to just gets ignored and you play the weirdly smooth, denoised video instead.
Desktop players
Well, I hit my first roadblock directly after my first test encode. I use mpvnet as my player of choice. Turns out grain synthesis is broken for me with the default configuration. After like an hour of debugging, I found that I could fix it by adding vo=gpu instead of the default vo=gpu-next in my config file. This may or may not mean that the grain filter is rendered by the CPU instead of the GPU, affecting performance.
VLC worked out of the box for me. Not sure if that was because it was using CPU grain rendering or if it was because their drivers were less broken.
Neither play has a clear option to turn on or off grain synthesis. In theory, you could configure mpv with format:film-grain=no, but that didn't seem to work for me. It's possible that option only works when GPU grain rendering is in use.
For reference, I use a 3000-series Nvidia GPU. I expect GPU rendering support may depend on what GPU you have. Perhaps it works better with more recent GPUs.
Plex
I placed the AV1 file from the experiment on my fully updated Plex server, and tried to play back the episode from a variety of different devices.
- Desktop app - AV1 and grain synthesis both worked
- Mobile app (Pixel 9 Pro) - AV1 and grain synthesis both worked
- Firefox - AV1 not supported. Transcoded to h264.
- Chrome - AV1 not supported. Transcoded to h264.
- Chromecast (1st-gen) - AV1 not supported. Transcoded to h264.
When transcoding occurred, the grain-synthesis would be ignored. Meaning that you'd get a transcode of the post-denoise video into h264. No / little grain (depending on how strongly you denoised).
This is somewhat worse than transcoding h265 into h264 for compatibility reasons, but at a high bitrate where the grain is still present. That said, on-the-fly transcoding rarely results in a particularly good looking outcome.
Conclusion
Both encodes turned out very watchable on compatible devices. There was no clear winner over h265 but AV1 + grain synthesis looks decent when it works.
That said, if you or others watch your video on unsupported devices, like on the browser or on your smart tv / chromecast, you may want to opt for h265 instead. At least if that has to be transcoded, you will maintain your grain in the process.
Additionally, the effort you go through tuning AV1 encodes is somewhat higher than just chucking it through a h265 or h264 encoder. In my experience, without grain synthesis AV1 does less well with grainy content than h265 does. Not too surprising given that AV1 is primarily a web standard while h265 comes from a more disc media background.
Wild that you can serve anything with that hardware. Granted, static websites are basically just sending files over the wire.