Characterizing and Taming Resolution in Convolutional Neural Networks

Abstract

Image resolution has a significant effect on the accuracy and computational, storage, and bandwidth costs of computer vision model inference. These costs are exacerbated when scaling out models to large inference serving systems and make image resolution an attractive target for optimization. However, the choice of resolution inherently introduces additional tightly coupled choices, such as image crop size, image detail, and compute kernel implementation that impact computational, storage, and bandwidth costs. Further complicating this setting, the optimal choices from the perspective of these metrics are highly dependent on the dataset and problem scenario. We characterize this tradeoff space, quantitatively studying the accuracy and efficiency tradeoff via systematic and automated tuning of image resolution, image quality and convolutional neural network operators. With the insights from this study, we propose a dynamic resolution mechanism that removes the need to statically choose a resolution ahead of time. Our evaluation shows that our dynamic resolution approach improves inference latency by 1.2×−1.7×, reduces data access volume by up to 20-30%, without affecting accuracy. We establish the dynamic resolution approach as a viable alternative to fine-tuning for a specific object scale to compensate for unknown crop sizes, which is the current state of the art.