The main goal of this article is to construct "arithmetic Okounkov bodies" for an arbitrary pseudo-effective (1,1)-class α on a Kähler manifold. First, we prove the differentiability of volumes of big classes for Kähler manifolds on which modified nef cones and nef cones coincide. As a consequence, we prove Demailly's transcendental Morse inequality for these particular Kähler manifolds. In the second part, we construct the generalized Okounkov body for any big (1,1)-class, and prove that it coincides with the Okounkov body when the big class is rational. Next, we give a complete characterization of generalized Okounkov bodies on surfaces, and relate the standard Euclidean volume of the body to the volume of the corresponding big class as defined by Boucksom; this solves a problem raised by Lazarsfeld and Mustaţȃ in the case of surfaces. Finally, we study the behavior of the generalized Okounkov bodies on the boundary of the big cones.