Redução de Complexidade no Affine Motion Estimation do Padrão VVC

Abstract

The current state-of-the-art of video coding is Versatile Video Coding (VVC), which developed the purpose of the growing need for video data compression resulting from the increased consumption of these contents. In addition, it brings versatility, with the proposal to support various video contents, including the news that is emerging in the market, such as virtual reality, augmented reality, and 360o videos, among others, also enabling support for high resolutions beyond 4k UHD. In 2015 a group of video encoding experts called the Joint Video Exploration Team (JVET) came together to create a new video encoder standard. So in July 2020, the new VVC encoding standard was finalized, aiming to achieve a 50% bitrate reduction over its predecessor High-Efficiency Video Coding (HEVC). VVC was implemented based on its predecessor HEVC, from which it inherited most of the tools, some were improved, and others added, enabling a reduction in the average bitrate compared to its predecessor of 44%. However, these improvements came with a consequence, the increase in computational complexity, which makes it impossible for this new standard to be used in certain situations, such as real-time video transmission or used in portable devices that have limited energy capacity. Affine is one of the new tools implemented in VVC to make a more efficient prediction of complex movements such as zooming, rotation, and shearing. However, the tool uses more sophisticated techniques, increasing the time to encode these movements, which may represent 54.75% of the total Motion Estimation (ME) time. Two proposals are presented, aiming at reducing the computational effort of the VVC, one hardware-friendly and the other in software optimization. The hardware-friendly proposal presents a configurable heuristic for the Unidirectional, Bidirectional, and Affine Motion Estimation steps of the VVC. The hardware-friendly heuristic is based on the adaptive evaluation of CU sizes to reduce power consumption. Three operation points are supported, achieving a reduction of between 7,7% and 30% in energy consumption, with a loss of image quality between 0,04% and 1%. The software-based proposal presents a fast Affine Motion Estimation (AME) for the VVC. This approach uses Machine Learning with the Random Forests (RF) classifier method, developing an RF model for each CU size. Information extracted from the current, parent, and neighbors CUs is used to train the models. Each model is used to define whether a given CU size should be ignored by AME or not. The proposed solution achieves an average time reduction of 20% in the AME and 3% in the total encoding time, with an image quality loss of 0,07%. Keywords: VVC, Inter-frames, Affine, Machine Learning, Random Forest