Boundary driven diffusive systems describe a broad range of transport phenomena. We study large deviations of the density profile in these systems, using numerical and analytical methods. We find that the large deviation may be non-differentiable, a phenomenon that is unique to nonequilibrium systems, and discuss the types of models which display such singularities. The structure of these singularities is found to generically be a cusp, which can be described by a Landau free energy or, equivalently, by catastrophe theory. Connections with analogous results in systems with finite-dimensional phase spaces are drawn.