Cognitive Radio (CR) technologies have been proposed as a promising solution to improve connectivity, selfadaptability, and efficiency of spectrum usage by using dynamic spectrum resource management. When used in multimedia applications in wireless networks, the end-to-end user-perceived video quality experienced by secondary users is a very important performance metric to evaluate the effectiveness of CR technologies. However, most of the current research activities consider only spectrum utilization and effectiveness, ignoring the system performance of upper layers. In this paper, we propose an end-to-end system to optimize the user-perceived video quality at the receiver end under the constraint of packet delay bound in cognitive wireless networks. By mathematically formulating the design objective and the design constraint, the impact of inextricably interrelated system parameters residing in different network layers are jointly considered and optimized in a holistic way to achieve the best-possible user-perceived video quality for secondary users. Further, we formulate the problem into a MIN-MAX problem, which is then solved by using dynamic programming. The performance enhancement by using the proposed system is demonstrated through extensive experiments based on H.264/AVC.