- 2.1.1OBS V23 - 19th February 2019
- 4.2How it works
- 4.4What to expect
- 5.3Conclusions for x264
- 6.3Conclusions for x265
- 7.3Conclusions for QuickSync
- 8NVENC part 1
- 8.3Special note for NVENC Part 1
- 8.4Conclusions for NVENC Part 1
- 8.4.2Decisions decisions...
- 8.4.3Final thoughts on Maxwell and Pascal H.264 AVC
- 9NVENC Part 2
- 9.3Conclusions for NVENC Part 2
- 9.3.1H.264 AVC and live streaming
- 9.3.2H.265 HEVC and offline encoding
- 10.3Conclusions for VP9
- 11.3Conclusions for AV1
- 12Final thoughts
For a decade and a half, x264 has been the go-to free open-source encoder for AVC H.264 video. It is the sequel to H.263, the format encoded by DivX and Xvid. Netflix and YouTube still use H.264 to a degree and x264 is the most common encoder used by streamers.
Up until recently, the best way to get a better quality video into limited bandwidth was to use x264. Improving CPU power is the best way to improve your x264 quality-per-bitrate. There are a few reasons for this:
- Original Kepler NVENC and Sandy Bridge QuickSync were very rudimentary in terms of quality, and only excelled in speed.
- Maxwell and Pascal NVENC had some new options for quality. However, not all of these are available for use in OBS without complicated custom options.
- x264 had nearly a decade of development before NVENC and QuickSync came out. x264 was very well optimised by comparison.
I’ve limited x264 encode presets to those that are reasonable on current CPUs for real-time streaming. Fortunately, going slower than Medium provides barely any benefit. Even at VerySlow the H.264 parameters find quality differently to VMAF and end up wasting bits. This is a consequence of how old the format is.
Bear in mind that NVENC for Pascal and Maxwell cards have a “slow” and “hq” version for each bitrate. OBS cannot access slow versions of NVENC by default. So comparing for streaming quality on this test video of 1440p 60 fps, x264:
- Beats Kepler, Maxwell and Pascal NVENC on Medium preset at 4, 6 & 8 Mbps. Medium preset x264 is Twitch’s default anchor encoder when examining alternatives.
- Beats Kepler, Maxwell and Pascal NVENC on Faster preset at 6Mbps.
- Always beats Kepler NVENC. Always loses to Turing NVENC, H.265, VP9 and AV1.
The situation with non-live encoding is a little different. Handbrake can encode using NVENC on the “slow” setting, which give Pascal and Maxwell a boost in quality per bitrate. However, Handbrake and FFMPEG have access to a plethora of x264 parameters. Even with Handbrake’s GUI you can access slower presets and tuning options. FFMPEG offers heaps more options for x264, there’s tonnes more customisation past simple presets. Depending on your video, you can change the parameters accordingly and optimise your x264 usage.
Conclusions for x264
When streaming, if you have a good CPU your best bet is likely x264. The downside is that if your gameplay uses a lot of CPU, it could take a hit. If you have an RTX 2080TI, 2080, 2070 or 2060 then Turing is invariably the better option. Twitch still requires H.264 video input (with some exceptions for VP9) so for now this will remain commonplace. However, YouTube already allows for streaming in VP9 depending on your channel. The near future suggests Twitch will begin allowing more codecs as video resolutions get larger.
For offline encoding you need to prioritise. If you need everything done quicker, then hardware encoder options will smash x264. Should you desire better compression, the H.265 or VP9 encoders will give better results. If you’re stuck with x264 then make sure you tune it to the video type you want.