We investigate a biologically motivated approach to fast visual classification, directly inspired by the recent work [13]. Specifically, trading-off biological accuracy for computational efficiency, we explore using standard wavelet transforms and patch transforms to parallel the tuning of visual cortex V1 and V4 cells, alternated with max operations to achieve scale and translation invariance. A feature selection procedure is applied during learning to accelerate recognition. We introduce a simple attention-like feedback mechanism, significantly improving recognition and robustness in multiple-object scenes. In experiments, the proposed algorithm achieves or exceeds state-of-the-art performance in object recognition, but also in new applications such as texture classification, satellite image classification, and language identification. Preliminary results on sound classification are shown as well.