First, we establish the theory of fractional powers of first order differential operators with zero order terms, obtaining PDE properties and analyzing the corresponding fractional Sobolev spaces. In particular, our study shows that Lebesgue and Sobolev spaces with inverse measures (like the inverse Gaussian measure) play a fundamental role in the theory of fractional powers of the first order operators. Second, and motivated in part by such a theory, we lay out the foundations for the development of the harmonic analysis for inverse measures. We discover new families of polynomials related to the inverse Gaussian, Laguerre, and Jacobi measures, and characterize them using generating and Rodrigues formulas, and three-term recurrence relations. Moreover, we prove boundedness of several fundamental singular integral operators in these inverse measure settings.