Block-based motion modelling has been widely used in video coding where a frame is divided into fixed-sized blocks that are motion compensated independently. This often leads to coding inefficiency as fixed-sized blocks hardly align with the object motion. Although hierarchical blockpartitioning has been introduced to address this, the increased number of motion vectors limits the benefit. Recently, approximate segmentation of images with cuboidal partitioning has gained popularity. Not only are the variable-sized rectangular segments (cuboids) readily amenable to block-based image/video coding techniques, but they are also capable of aligning well with the object boundaries. This is because cuboidal partitioning is based on a homogeneity constraint, minimising the sum of squared errors (SSE). Therefore, motion modelling with variable-sized cuboidal blocks is able to exploit semantic correlations by following displacements of approximate object boundaries. The existing cuboidal partitioning-based method generates a reference frame using the mean value of the cuboids but they didn’t exploit any motion information. In this paper, we have investigated the potential of cuboids in motion modelling against the fixed-sized blocks used in scalable video coding. Specifically, we have constructed motioncompensated current frame using the cuboidal partitioning information of the anchor frame in a group-of-picture (GOP). The predicted current frames have then been used as the base layer while encoding the current frames as an enhancement layer using a scalable encoder. We have also proposed two novel approaches to determine the number of cuboids by which each frame will be partitioned to. Experimental results confirm high bitrate savings on a wide range of video sequences.