Although it is well known that solid particles adsorb at interfaces, no consensus has been reached on whether or not the adsorbed nanoparticles affect interfacial tension. In this work the Wilhelmy plate method is implemented in mesoscale dissipative particle dynamics (DPD) simulations to study the influence of nanoparticles on the water-oil interfacial tension. The results are compared with predictions that neglect nanoparticlenanoparticle interactions at the interface. We find that the two estimates can differ significantly. In the regime where nanoparticle-nanoparticle repulsion is large, the Wilhelmy plate method suggests interfacial tension reduction, which appears to be a strong function of nanoparticle surface coverage. Some experimental data from the literature, in apparent disagreement, are re-interpreted based on this insight.