In the present work we introduce the concept of solitary wave billiards. I.e., instead of a point particle, we consider a solitary wave in an enclosed region and explore its collision with the boundaries and the resulting trajectories in cases which for particle billiards are known to be integrable and for cases that are known to be chaotic. A principal conclusion is that solitary wave billiards are generically found to be chaotic even in cases where the classical particle billiards are integrable. However, the degree of resulting chaoticity depends on the particle speed and on the properties of the potential. Furthermore, the nature of the scattering of the deformable solitary wave particle is elucidated on the basis of a negative Goos-Hänchen effect which, in addition to a trajectory shift, also results in an effective shrinkage of the billiard domain.