In this paper, we give the determinant expressions of the hypergeometric Bernoulli numbers, and some relations between the hypergeometric and the classical Bernoulli numbers which include Kummer's congruences. By applying Trudi's formula, we have some different expressions and inversion relations. We also determine explicit forms of convergents of the generating function of the hypergeometric Bernoulli numbers, from which several identities for hypergeometric Bernoulli numbers are given.