Groupwise registration has been widely investigated in recent years due to its importance in analyzing population data in many clinical applications. To our best knowledge, most of the groupwise registration algorithms only utilize the intensity information. However, it is well known that using intensity only is not sufficient to achieve the anatomically sound correspondences in medical image registration. In this article, we propose a novel feature-based groupwise registration algorithm to establish the anatomical correspondence across subjects by using the attribute vector that is defined as the morphological signature for each voxel. Similar to most of the state-of-the-art groupwise registration algorithms, which simultaneously estimate the transformation fields for all subjects, we develop an energy function to minimize the intersubject discrepancies on anatomical structures and drive all subjects toward the hidden common space. To make the algorithm efficient and robust, we decouple the complex groupwise registration problem into two easy-to-solve subproblems, namely (1) robust correspondence detection and (2) dense transformation field estimation, which are systematically integrated into a unified framework. To achieve the robust correspondences in the step (1), several strategies are adopted. First, the procedure of feature matching is evaluated within a neighborhood, rather than only on a single voxel. Second, the driving voxels with distinctive image features are designed to drive the transformations of other nondriving voxels. Third, we take advantage of soft correspondence assignment not only in the spatial domain but also across the population of subjects. Specifically, multiple correspondences are allowed to alleviate the ambiguity in establishing correspondences w.r.t. a particular subject and also the contributions from different subjects are dynamically controlled throughout the registration. Eventually in the step (2), based on the correspondences established for the driving voxels, thin-plate spline is used to propagate correspondences on the driving voxels to other locations in the image. By iteratively repeating correspondence detection and dense deformation estimation, all the subjects will be aligned onto the common space. Our feature-based groupwise registration algorithm has been extensively evaluated over 18 elderly brains, 16 brains from NIREP (with 32 manually delineated labels), 40 brains from LONI LPBA40 (with 54 manually delineated labels), and 12 pairs of normal controls and simulated atrophic brain images. In all experiments, our algorithm achieves more robust and accurate registration results, compared with another groupwise algorithm and a pairwise registration method.