The determination of the wall thickness [intima-media thickness (IMT)], the delineation of the atherosclerotic carotid plaque, the measurement of the diameter in the common carotid artery (CCA), as well as the grading of its stenosis are important for the evaluation of the atherosclerosis disease. All these measurements are also considered to be significant markers for the clinical evaluation of the risk of stroke. A number of CCA segmentation techniques have been proposed in the last few years either for the segmentation of the intima-media complex (IMC), the lumen of the CCA, or for the atherosclerotic carotid plaque from ultrasound images or videos of the CCA. The present review study proposes and discusses the methods and systems introduced so far in the literature for performing automated or semi-automated segmentation in ultrasound images or videos of the CCA. These are based on edge detection, active contours, level sets, dynamic programming, local statistics, Hough transform, statistical modeling, neural networks, and an integration of the above methods. Furthermore, the performance of these systems is evaluated and discussed based on various evaluation metrics. We finally propose the best performing method that can be used for the segmentation of the IMC and the atherosclerotic carotid plaque in ultrasound images and videos. We end the present review study with a discussion of the different image and video CCA segmentation techniques, future perspectives, and further extension of these techniques to ultrasound video segmentation and wall tracking of the CCA. Future work on the segmentation of the CCA will be focused on the development of integrated segmentation systems for the complete segmentation of the CCA as well as the segmentation and motion analysis of the plaque and or the IMC from ultrasound video sequences of the CCA. These systems will improve the evaluation, follow up, and treatment of patients affected by advanced atherosclerosis disease conditions.