Naturally occurring sounds are routinely periodic. The ability to phase-lock to such periodicity facilitates pitch perception and interaural time differences (ITDs) determination in binaural localization. We examined whether deficient pitch processing in individuals with congenital amusia (tone deafness) is accompanied by impaired ability to lateralize musical pitch at auditory periphery and memorize the location of pitch at the working memory level. If common mechanisms subserve processing of temporal-fine-structure based pitch and ITDs, then deficient processing of one feature should impair performance on the other. Thus, we measured ITD discrimination thresholds using an adaptive-tracking procedure for lateralizing musical tone pairs separated by different semitone intervals. Amusic individuals exhibited normal ITD thresholds comparable to those of matched controls, which were not affected by concurrent pitch changes. For working memory tasks, the amusic group performed significantly worse than matched controls in probed pitch recall, irrespective of the complexity level of concurrent variations along the ITD dimension of the melodic sequence. Interestingly, despite normal peripheral ITD thresholds, amusic individuals performed worse than controls in recalling probed locations of tones within a sequence of musical notes originating from different ITD-simulated locations. Findings suggest that individuals with congenital amusia are unimpaired in temporal fine-structure encoding to determine the location of musical pitch based on binaural ITD information at the auditory periphery. However, working memory for a sequence of sounds’ ITD-dependent spatial location is here shown to be impaired and dissociated from the pitch feature of sounds at the working memory level.