We have studied the specific heat of the (N S + N B ) model for an N S -body harmonic oscillator (HO) system which is strongly coupled to an N B -body HO bath without dissipation. The system specific heat of C S (T ) becomes N S k B at T → ∞ and vanishes at T = 0 in accordance with the third law of thermodynamics. The calculated C S (T ) at low temperatures is not proportional to N S and shows an anomalous temperature dependence, strongly depending on N S , N B and the system-bath coupling. In particular at very low (but finite) temperatures, it may become negative for a strong system-bath coupling, which is in contrast with non-negative specific heat of an HO system with N S = 1 reported by G-L. Ingold, P. Hänggi and P. Talkner [Phys. Rev. E 79, 061105 (2005)]. Our calculation indicates an importance of taking account of finite N S in studying open quantum systems which may include an arbitrary number of particles in general.