Using machine learning for fast intra MB coding in H.264

Abstract

H.264 is a highly efficient and complex video codec. The complexity of the codec makes it difficult to use all its features in resource constrained mobile devices. This paper presents a machine learning approach to reducing the complexity of Intra encoding in H.264. Determining the macro block coding mode requires substantial computational resources in H.264 video encoding. The goal of this work to reduce MB mode computation from a search operation, as is done in the encoders today, to a computation. We have developed a methodology based on machine learning that computes the MB coding mode instead of searching for the best match thus reducing the complexity of Intra 16x16 coding by 17 times and Intra 4x4 MB coding by 12.5 times. The proposed approach uses simple mean value metrics at the block level to characterize the coding complexity of a macro block. A generic J4.8 classifier is used to build the decision trees to quickly determine the mode. We present a methodology for Intra MB coding. The results show that intra MB mode can be determined with over 90% accuracy. The proposed can also be used for determining MB prediction modes with an accuracy varying between 70% and 80%. INTRODUCTIONRecent developments in video encoding such as the H.264 and VC1 have resulted in highly efficient compression. These new generation codecs are highly efficient and result in equivalent quality video at 1/3 to ½ of MPEG-2 video bitrates. The complexity of this new encoder, however, is 10 times as complex [1]. The compression efficiency has a high computational cost associated with it. The high computational cost is the key reason why these increased compression efficiencies cannot be exploited across all application domains. Resource constrained devices such as cell phones, embedded cameras, and video sensors use simpler encoders or simpler profiles of new codecs to tradeoff compression efficiency and quality for reduced complexity. The new video codecs from Microsoft and Real are also based on hybrid coding techniques similar to H.264 and are comparable in complexity and quality.The compressions efficiency of these new codecs has increased mainly because of the large number of coding options available. For example, the H.264 video supports Intra prediction with 3 different block sizes and Inter prediction with 8 different block sizes. The encoding of a macro block (MB) involves evaluating all the possible block sizes. As the number of reference frames is increased, the complexity increases proportionally. New approaches are necessary to drastically reduce the encoding complexity without sacrificing quality. We are developing low complexity encoding tools based on machine learning techniques. The goal is to reduce MB mode and motion vector search to a decision tree with a motion search in a small window. The proposed approach is based on the hypothesis that video frames can be characterized for the purpose of encoding and the encoding complexity can be drastically reduced. Our work on MPEG-2 to H.264 transcoding has shown that ma...

Video coding mode decision as a classification problem

“…Analytical results based on this approach show that the number of operations required for Intra mode determination is reduced by about 15 times. 7 We implemented the decision tree in order to evaluate the performance of the chance constrained based decisions. The decision trees were implemented in H.264 reference software JM 14.2.…”

Fast H.264/AVC FRExt Intra Coding Using Belief Propagation

“…This makes the computational complexity vary significantly according to the processed video sequence (see [11] as an example where the relative reduction of coding time varies from 40% to 70%), and therefore, an "a priori" estimation of the resulting cost is not possible. At the same time, the performance of the algorithm varies according to the coded sequence like in the case of [12] where a machine learning algorithm is used to select the best prediction mode among a reduced set of candidates.…”

Spatial Prediction in the H.264/AVC FRExt Coder and its Optimization

“…The same policy is applied to the new candidate modes following a tree-ordered refinement policy of the Intra prediction for the current block. Another hierarchical solution was proposed by Kalva and Christodoulou (2007), where the modes are tested following an adaptive tree structure that is modified using a machine learning algorithm.…”

“…This makes the computational complexity significantly vary according to the processed video sequence (see the strategy proposed by Yong-dong et al (2004) as an example where the relative reduction of coding time varies from 40% to 70%), and therefore, an a priori estimation of the resulting cost is not possible. At the same time, the performance of the algorithm varies according to the coded sequence like in the case of the approach by Kalva and Christodoulou (2007) where a machine learning algorithm is used to select the best prediction mode among a reduced set of candidates. With respect to these methods, the design of a complexity reduction strategy that permits controlling the amount of required computation provides several advantages, such as…”