In recent years, video delivery over wireless visual sensor networks (VSNs) has g ained increasin g attention. The lossy compression and channel errors that occur durin g wire less multimedia transmissions can de g rade the quality of the transmitted video sequences. This paper addresses the problem of cross-layer resource allocation amon g the nodes of a wireless direct-sequence code division multiple access (DS-CDMA) VSN.The optimal g roup of pictures (GoP) len g th durin g the encodin g process is also considered, based on the motion level of each video sequence. Three optimization criteria that optimize a different objective function of the video qualities of the nodes are used. The nodes' transmission parameters, i.e., the source codin g rates, channel codin g rates and power levels can only take discrete values. In order to tackle the resultin g optimization problem, a reinforcement learning (RL) strate g y that promises efficient exploration and exploitation of the parameters' space is employed. This makes the proposed methodolo g y usable in lar g e or continuous state spaces as well as in an online mode.Experimental results hi g hli g ht the efficiency of the proposed method.