Video Quality Measurement & Control for Live Encoding

Video Quality (VQ) Measurement is essential in a variety of applications, e.g. to compare video codecs, to select bitrates or configurations, or to optimize encoder behavior. Many different VQ metrics have been developed over the past decades, both full-reference (comparing source and encoded videos) and no-reference (without access to the source) [3]. For most offline encoding use cases, these metrics typically suffice, and give a fairly accurate view of the subjective quality of compressed video streams. In live video distribution, however, VQ measurement becomes a lot more challenging, in particular when computational and cost constraints come into play. While several accurate metrics exist (such as MS-SSIM and VMAF), these are often too complex for real-time video compression and decision making. Faster metrics exist, but these typically lack accuracy.

Fortunately, we have seen initiatives towards more efficient metrics. As an example, simplifications of SSIM have been introduced in [1] In recent years, we have seen efforts to reduce the computational complexity of VMAF [2]. By using vectorization, fixed-point approximations, and multi-threading, the runtime of VMAF has already been reduced. Still, its overall complexity remains high. In particular, the complexity relative to that of the encoding process is an important driver for its usage. Translated into business terms, the cost per channel of the VQ metric becomes an important criterion for adoption, and can be prohibitively high.

In this talk, we focus on the computational complexity of commonly used VQ metrics, and discuss approximations for further complexity reduction, including approaches based on machine learning. We address the difficulties around real-time VQ measurement from a complexity/cost standpoint, especially when these metrics have to be embedded inside live encoders.

Furthermore, we introduce which metrics can be used inside encoders for real-time decision making and active video quality control. Based on the fast quality metrics discussed above, advanced rate-quality control mechanisms can be embedded deep inside the encoders. Quality control is beneficial in multiple scenarios, including: (i) for CBR quality control; (ii) for ABR quality control and profile optimization; and (iii) for intelligent bit distribution within "traditional" statmux bundles. Quality control leads to more efficient bitrate allocation, and more constant quality throughout the video stream. We introduce novel algorithms for quality control for each of these applications, along with compression savings we measured in the field.