We present an analysis of the galaxy stellar mass function in different environments at intermediate redshift (0.3 ≤ z ≤ 0.8) for two mass-limited galaxy samples. We use the IMACS Cluster Building Survey (ICBS; M * ≥ 10 10.5 M ) to study cluster, group and field galaxies at z = 0.3-0.45, and the ESO Distant Cluster Survey (EDisCS; M * ≥ 10 10.2 M ) to investigate cluster and group galaxies at z = 0.4-0.8. Our analysis thus includes galaxies with masses reaching just below that of the Milky Way. Excluding the brightest cluster galaxies, we show that the shape of the mass distribution does not seem to depend on global environment, Our two main results are: (1) Galaxies in the virialised regions of clusters, in groups, and in the field follow similar mass distributions. (2) Comparing the ICBS and EDisCS mass functions to mass functions in the local universe, we detect evolution from z ∼ 0.4-0.6 to z ∼ 0.07 in the sense that the population of low-mass galaxies has grown with time with respect to the population of massive galaxies. This evolution is independent of environment, i.e., the same for clusters and the field. Furthermore, considering only cluster galaxies, we find that no differences can be detected in their mass functions either within the virialised regions, or when we compare galaxies inside and outside the virial radius. Finally, we find that red and blue galaxies have different mass functions. However, the shapes of the mass functions of blue and red galaxies do not seem to depend on their environment (clusters groups and the field).