This paper presents a novel skeleton-based audio envelope representation and matching method for audio signal analysis. We propose using amplitude envelope in the time domain to represent and calculate the similarity between audio signals. To effectively describe the shape of each envelope, we employ the skeleton descriptor, namely Audio Skeleton, to integrate both geometrical and topological envelope features. Based on Audio Skeletons, the audio envelope matching can be substituted by searching for the correspondences of skeleton endpoints. Finally, the similarity between audio envelope shapes is calculated based on their correlated skeleton matching. Our main contributions include (i) the introduction of a skeleton-based audio envelope descriptor, (ii) a simple and efficient Audio Skeleton representation method and (iii) a fast skeleton pruning and matching algorithm.