We propose a new weakly supervised approach for classification and clustering based on mixture models. Ourapproach integrates multi-level pairwise group and classconstraints between samples to learn the underlyinggroup structure of the data and propagate (scarce) initial labels to unlabelled data. Our algorithm assumes thenumber of classes is known but does not assume anyprior knowledge about the number of mixture components in each class. Therefore, our model : (1) allocatesmultiple mixture components to individual classes, (2)estimates automatically the number of components ofeach class, 3) propagates class labels to unlabelled datain a consistent way to predefined constraints. Experiments on several real-world and synthetic data datasetsshow the robustness and performance of our model overstate-of-the-art methods.