Determining the optimal focal plane amongst a stack of blurred images in a short response time is a non-trivial task in optical imaging like microscopy and photography. An autofocusing algorithm, or in other words, a focus metric, is key to effectively dealing with such problem. In previous work, we proposed a structure tensor-based autofocusing algorithm for coherent imaging, i.e., digital holography. In this paper, we further extend the realm of this method in more imaging modalities. With an optimized computation scheme of structure tensor, a significant acceleration of about fivefold in computation speed without sacrificing the autofocusing accuracy is achieved by using the Schatten matrix norm instead of the vector norm. Besides, we also demonstrate its edge extraction capability by retrieving the intermediate tensor image. Synthesized and experimental data acquired in various imaging scenarios such as incoherent microscopy and photography are demonstrated to verify the efficacy of this method.