We consider algebras that can be realized as PBW deformations of (Artin-Schelter) regular algebras. This is equivalent to the homogenization of the algebra being regular. It is shown that the homogenization, when it is a geometric algebra, contains a component whose points are in 1-1 correspondence with the simple modules of the deformation. We classify all PBW deformations of 2-dimensional regular algebras and give examples of 3-dimensional deformations. Other properties, such as the skew Calabi-Yau property and closure under tensor products, are considered.