Software Designs of Image Processing Tasks With Incremental Refinement of Computation

Abstract

We propose software designs that perform incremental computation with monotonic distortion reduction for twodimensional convolution and frame-by-frame block-matching tasks. In order to reduce the run time of the proposed designs, we combine bitplane-based computation with a packing technique proposed recently. In the case of block matching, we also utilize previously-computed motion vectors to perform localized search when incrementing the precision of the input video frames. The applicability of the proposed approach is demonstrated by execution time measurements on the xo-laptop ("100$ laptop") and on a mainstream laptop; our software is also made available online. In comparison to the conventional (non-incremental) software realization, the proposed approach leads to scalable computation per input frame while producing identical (or comparable) precision for the output results of each operating point. In addition, the execution of the proposed designs can be arbitrarily terminated for each frame with the output being available at the already-computed precision.Index Terms-complexity-scalable image processing, incremental refinement of computation, programmable processors INTRODUCTIONSeveral popular applications, such as media players, image and video post-processing, and motion estimation and compensation, are being implemented today via software solutions in general-purpose processors. New generations of processors are increasingly powerful and enable more dedicated resource allocation to such real-time multimedia tasks due to multicore designs [1]. At the same time, new generations of software compilers now automatically generate platform-specific optimized assembly code from C++ code [2], thereby enabling platform-independent C++ software solutions to achieve high processor utilization factors. Existing algorithmic-oriented research focuses on complexity reduction [3]- [5] or complexity scalability for image processing tasks [6]-[8], where computational complexity is decreased and approximate results are produced. Implementation-oriented research focuses on multimediadriven energy scaling of processors via dynamic voltage scaling [9] [10] in an attempt to provide computational scalability with approximate results. However, previous approaches can only obtain one operational point in the complexity-distortion curve [3] [7], without being able to increment the quality of the output with increased computation. In addition, in practical image and video coding systems, complexity does not scale down significantly with decreased source precision (decreased bitrate) [7]. Finally, existing image processing realizations produce an "all or nothing" output: one cannot interrupt the computation when system resources suddenly become unavailable (or when delay constraints are about to be violated) and retrieve a meaningful approximation of the final result1 . An exception is found in proposals for incremental computation of transforms and salient point detection algorithms [11] [12], where the main principle is:...

Prediction-Based Incremental Refinement for Binomially-Factorized Discrete Wavelet Transforms

“…This can reduce the expected overhead when increasing the number of increment layers. In our recent work on software designs for incremental image processing [6], we found that three or four increment layers provide for sufficient quality-complexity scalability without significant overhead in the overall execution time. From the implementation perspective, a crucial element needed in order to take advantage of the reduction of the multiplication bitwidth is a multiplier unit with adjustable circuit activity according to the input bit patterns.…”

“…However, since (19) and (20) involve two multiplications, this may not be cost-effective in comparison to (6).…”

“…For this reason, previous work proposed schemes for approximate computation of transforms and signal processing operations [3]. A property that has been recognized to be of great importance is incremental refinement of computation [4]- [6], where the transform representation of a signal (image, video) is produced incrementally with the use of embedded (bitplane-based) quantization. In our recent work [4], this design has been theoretically analyzed both for the forward and the inverse two-dimensional multilevel DWT using the generalization of the spatial combinative lifting algorithm (SCLA) of Meng and Wang [7].…”

Linear Image Processing Operations With Operational Tight Packing

“…ACKED linear image processing hinges on the idea that the dynamic range of a 32-bit or 64-bit numerical representation can be used for the concurrent calculation of multiple small-dynamic-range integer operations if the operands are positioned (or "packed") in such numerical representation with appropriate spacing from each other [1], [2]. This has been proposed for a variety of image processing operations such as bound estimation, image cross-correlation and orientation correlation [1], incremental image convolution and motion estimation [2], integer block-transform decomposition [3] and integer wavelet transforms [4].…”

“…This has been proposed for a variety of image processing operations such as bound estimation, image cross-correlation and orientation correlation [1], incremental image convolution and motion estimation [2], integer block-transform decomposition [3] and integer wavelet transforms [4].…”