Amorphous boron carbon nitride (a-BCN) films exhibit excellent properties such as high hardness and high wear resistance. However, the correlation between the film structure and its mechanical properties is not fully understood. In this study, a-BCN films were prepared by an arc-sputtering hybrid process under various coating conditions, and the correlations between the film’s structure and mechanical properties were clarified. Glow discharge optical emission spectroscopy, X-ray photoelectron spectroscopy, Fourier-transform infrared spectroscopy, and Raman spectroscopy were used to analyze the structural properties and chemical composition. Nanoindentation and ball-on-disc tests were performed to evaluate the hardness and to estimate the friction coefficient and wear volume, respectively. The results indicated that the mechanical properties strongly depend on the carbon content in the film; it decreases significantly when the carbon content is <90%. On the other hand, by controlling the contents of boron and nitrogen to a very small amount (up to 2.5 at.%), it is possible to synthesize a film that has nearly the same hardness and friction coefficient as those of an amorphous carbon (a-C) film and better wear resistance than the a-C film.