Molecular dynamics simulations have been carried out to study the interaction between two point defects in III-nitride materials (AlN, GaN and InN): a substitutional metal atom (Indium, Aluminum, Gallium) and the N-vacancy. The Stillinger-Weber (S-W) empirical potential is used. By calculating the potential energies of different configurations with these two defects, it is shown that the indium atom in AlN and GaN or aluminum and gallium in InN are stable in the immediate vicinity of the N-vacancy. In contrast, the gallium atom in AlN and the aluminum atom in GaN may be difficult to bring near the N-Vacancy. This behavior is related to the stress relaxation in the presence of these point defects. In AlN, the stability of indium atoms around the N-vacancy is the highest, indicating a good probability of aggregation, which may constitute a first explanation for the reported phase segregation during the growth of InAlN alloys.