In this work, we study the emotional link between music and color as well as the relationship that pleasure and familiarity have with music-evoked emotions, emphasizing on what happens with atonal music. This is important, since it is already known the role of music's mode (major or minor) on determining the perception of musical emotion for tonal music and, thus, its association with color. We decided to use Russell's (1980) multidimensional emotional scale model to assess music and colors, together with the four dimensions of the perceptual characteristics of color (Red-green, yellow-blue, saturation, and brightness) used by Palmer et al. (2013). Our results showed that atonal music shares a behavior similar to that found by Palmer et al. (2013) for tonal music, specifically, that music evaluated as joyful is linked to the colors perceived with that same emotion (saturated, light, and yellow), and sad music is linked to sad colors (desaturated, dark, and blue). Therefore, these findings reaffirm the importance of emotion on mediating the music-to-color association in both tonal and atonal music. Regarding our other variables, we found a positive correlation between pleasure and familiarity as well as between pleasure and emotional valence, although half of the music evaluated as sad (negative valence) was also evaluated as pleasant.