The computation of the model parameters of a canonical polyadic decomposition (CPD), also known as the parallel factor (PARAFAC) or canonical decomposition (CANDECOMP) or CP decomposition, is typically done by resorting to iterative algorithms, e.g. either iterative alternating least squares type or descent methods. In many practical problems involving tensor decompositions such as signal processing, some of the matrix factors are banded. First, we develop methods for the computation of CPDs with one banded matrix factor. It results in best rank-1 tensor approximation problems. Second, we propose methods to compute CPDs with more than one banded matrix factor. Third, we extend the developed methods to also handle banded and structured matrix factors such as Hankel or Toeplitz. Computer results are also reported.