Motivated by the recently synthesized insulating nickelate Ni 2 Mo 3 O 8 , which has been reported to have an unusual noncollinear magnetic order of Ni 2+ S = 1 moments with a nontrivial angle between adjacent spins, we construct an effective spin-1 model on the honeycomb lattice, with the exchange parameters determined with the help of first-principles electronic-structure calculations. The resulting bilinear-biquadratic model, supplemented with the realistic crystal-field induced anisotropy, favors the collinear Néel state. We find that the crucial key to explaining the observed noncollinear spin structure is the inclusion of the Dzyaloshinskii-Moriya (DM) interaction between the neighboring spins. By performing variational mean-field and linear spin-wave theory (LSWT) calculations, we determine that a realistic value of the DM interaction D ≈ 2.78 meV is sufficient to quantitatively explain the observed angle between the neighboring spins. We furthermore compute the spectrum of magnetic excitations within the LSWT and random-phase approximation, which should be compared to future inelastic neutron measurements.