H.264/AVC standard uses rate-distortion optimization framework with heavy calculation and full search algorithm to select the best mode of each macroblock. Besides, 8×8 luma block prediction is added to high profile, which makes the search algorithm process more complicated. In order to reduce the amount of calculation, a filter method based on image texture feature and image complexity is adopted in this paper. Experiment results show that the new algorithm achieves an average saving of 43%-57% in encoding time ,with negligible effect on bitrate and PSNR.

This paper proposes a scheme of H.264 high-definition encoder/transcoder . The system is based on ASIC. Application-specific high-definition encoding/transcoding chip, software control module, HDMI interface, analog video/audio input interface and ASI input/output interface ,etc. are discussed in detail in this paper. The system also provides the Ethernet interface, which is used for real-time monitoring of coding/transcoding system . The proposed scheme with the property of good real-time and low cost accord with the need of high-definition digital TV front-end system.