Identification of material properties of hyper-elastic materials such as soft tissues of the human body or rubber-like materials has been the subject of many works in recent decades. Boundary conditions generally play an important role in solving an inverse problem for material identification, while their knowledge has been taken for granted. In reality, however, boundary conditions may not be available on parts of the problem domain such as for an engineering part, e.g., a polymer that could be modeled as a hyper-elastic material, mounted on a system or an in vivo soft tissue. In these cases, using hypothetical boundary conditions will yield misleading results. In this paper, an inverse algorithm for the characterization of hyper-elastic material properties is developed, which takes into consideration unknown conditions on a part of the boundary. A cost function based on measured and calculated displacements is defined and is minimized using the Gauss–Newton method. A sensitivity analysis is carried out by employing analytic differentiation and using the finite element method (FEM). The effectiveness of the proposed method is demonstrated through numerical and experimental examples. The novel method is tested with a neo–Hookean and a Mooney–Rivlin hyper-elastic material model. In the experimental example, the material parameters of a silicone based specimen with unknown boundary condition are evaluated. In all the examples, the obtained results are verified and it is observed that the results are satisfactory and reliable.