We introduce the Hopf algebra of uniform block permutations and show that it is self-dual, free, and cofree. These results are closely related to the fact that uniform block permutations form a factorizable inverse monoid. This Hopf algebra contains the Hopf algebra of permutations of Malvenuto and Reutenauer and the Hopf algebra of symmetric functions in non-commuting variables of Gebhard, Rosas, and Sagan. These two embeddings correspond to the factorization of a uniform block permutation as a product of an invertible element and an idempotent one.