Strongly regular graphs (SRGs) provide a fertile area of exploration in algebraic combinatorics, integrating techniques in graph theory, linear algebra, group theory, finite fields, finite geometry, and number theory. Of particular interest are those SRGs with a large automorphism group. If an automorphism group acts regularly (sharply transitively) on the vertices of the graph, then we may identify the graph with a subset of the group, a partial difference set (PDS), which allows us to apply techniques from group theory to examine the graph. Much of the work over the past four decades has concentrated on abelian PDSs using the powerful techniques of character theory. However, little work has been done on nonabelian PDSs. In this paper we point out the existence of genuinely nonabelian PDSs, that is, PDSs for parameter sets where a nonabelian group is the only possible regular automorphism group. We include methods for demonstrating that abelian PDSs are not possible for a particular set of parameters or for a particular SRG. Four infinite families of genuinely nonabelian PDSs are described, two of which—one arising from triangular graphs and one arising from Krein covers of complete graphs constructed by Godsil—are new. We also include a new nonabelian PDS found by computer search and present some possible future directions of research.