Encoder Comparison NVENC x264 Quicksync QSV VP9 and AV1

2018 was the year of the m.2 NVME drive. 2019 will be the year of hardware encoders like NVENC. I’ve used QHD 1440p 60fps Overwatch gameplay to test how a variety of codecs and encoders measure up using Netflix’s VMAF quality assessment algorithm. The most interesting development is the new NVENC implementation on Turing. This is available on the RTX 2080TI, 2080, 2070 and 2060.

There’s an older post with details on the nature of compression here. It also has some tips on using OBS, Handbrake and AVIDemux.

Introduction

There’s a few different situations to consider with encoders. One situation is streaming with OBS which has real-time encoding concerns. Another is a video serving system, where playback compatibility, licensing and saving bandwidth is paramount. Archiving is different again, where pristine quality is the primary concern and saving space is secondary for a huge collection.

Keep those things in mind when comparing codecs. This post is using Overwatch gameplay source video in 2560×1440 at 60 frames per second. The reason for this is to take things just a step higher than current standards. Most streamers on Twitch and YouTube use either 720p or 1080p and 30 or 60 fps depending on bandwidth. Most on-demand video is available in 1080p. The extreme position would be to use 2160p or 4k video for this comparison, but that would distance it too far from the current norm.

A good middle ground between current standard video sizes and what’s coming in the future is 1440p60. Some YouTube videos come in this size or higher, but most are lower. Twitch streams are almost exclusively smaller. Eventually 4k and 2160p will be the norm. For the immediate future, we can safely expect 1440p to be the next frontier. By the time 1440p is not a relevant resolution, there will likely be significant changes to some existing encoders as well as some new options to consider.

Disclaimer

The answer to a situation is not always as simple as one encoder or codec being “better” than the rest. Pay attention to the results for long enough and you’ll see where exceptions appear. Different encoders, especially hardware ones, suffer from bit starvation at one end of the scale. Then at the other end they may struggle to take advantage of bandwidth abundance. The easiest example to see is QuickSync. There is a sweet spot where QuickSync’s chipset can make great use of available bandwidth and compression. Too little bandwidth and it’s no longer optimised, and underperforms compared to it’s competitors. Too much bandwidth and it’s limitations are unable to efficiently take advantage of the extra bits. Sometimes, an encoder like QuickSync, is the winner for one bitrate but not for another.

One additional exception is CPU usage. You may be able to use one software encoder fine for Overwatch, but it will fail with Battlefield 5. The reason is that different games consume different amounts of processing power and your encoder can become starved for cycles. Simpler CPUs that are brand new in 2019 may cope quite well with codecs from 15 years ago. But then they suffer drastically with the more recent codecs like VP9 and AV1. Just because your brand new 9900k smashes x264 encoding does not mean it can cope with libvpx-vp9 at all. Sometimes a good Threadripper or 7940x from 2017 and intelligent usage is better than basic usage of a 2019 CPU.

Updates

I did all this testing in January. Throughout the year I’ll add updates here to check on the progress for 2019.

OBS V23 – 19th February 2019

OBS V23 will change the access to the NVENC encoder. It’s up to it’s second release candidate today on the 18th of February 2019 so expect it to launch soon. Essentially, the difference between the “hq” and “slow” presets in NVENC Part 1 will narrow in v23 of OBS. There will be a preset “Max Quality” that will provide closer to “slow” and it will be in the OBS preset drop-down. This adds some additional life to all pre-Turing NVENC generations. Turing itself does not require this change. It does work slightly differently, but in terms of VMAF scores, Turing will almost always work to it’s highest score on the graph, while Pascal and Maxwell actually need this change to improve their scores.

Contents

I’ve broken up this analysis into several pages.

  • VMAF, why use it for quality comparison and what to expect.
  • x264, the classic H.264 encoder in a few presets.
  • x265, the newer H.265 format in several presets.
  • QuickSync, in both H.264 and H.265 variants.
  • NVENC, including Kepler, Maxwell, Pascal and Turing series cards.
  • VP9, just one single config to compare to the rest.
  • AV1, the codec of the foreseeable future.

UNREAL AUSSIES ARE ALL OVER THE WEB

HELP SUPPORT
UNREAL AUSSIES

Unreal Aussies run many events over the year to help connect and build the Australian gaming community. If you are interested in helping out in any current or future planned events or wishing to offer some more ideas for us to explore - let us know!

About Us

Unreal Aussies is for passionate gamers from all walks of life. Games come and go, but the people still remain. From meetups to tournaments, hardcore teams to charity streams, Unreal Aussies core mission is to make gaming more fun as part of a community than it can ever be alone.