Electromagnetic wave chaos is investigated using two-dimensional optical cavities formed in a cylindrical gradient refractive index lens with reflective surfaces. When the planar ends of the lens are cut at an angle to its axis, the geometrical ray paths are chaotic. In this regime, the electromagnetic mode spectrum of the cavity is modulated by both real and ghost periodic ray paths, which also "scar" the electric field intensity distributions of many modes. When the cavity is coupled to waveguides, the eigenmodes generate complex series of resonant peaks in the electromagnetic transmission spectrum.