Analysis of performance characteristics of nanoparticles’ dispersed oil has wide and an important role in oil industries. Zinc oxide nanoparticles are dispersed with polyester oil by using an ultrasonic vibrator and magnetic stirrer. Nanoparticles’ concentration are taken as 0.1, 0.3, and 0.5 wt. %. The ball-milling process is used to reduce the size of the nanoparticles before the preparation of nanoparticles’ dispersed polyester oil (NPDP). Spherical shape particles with an average size of 40 nm are achieved for unmilled nanoparticles, whereas spherical shape particles with an average size of 30 nm are achieved for ball-milled nanoparticles. Pin on disc analysis shows the coefficient of friction of unmilled and ball-milled NPDP. Ball-milled NPDP results in a lesser coefficient of friction. Thermal conductivity of unmilled and ball-milled NPDP is finalized through the transient hot-wire method. Ball-milled NPDP enhances the thermal conductivity of base polyester oil than the usage of unmilled NPDP in polyester oil. In this study, the assessment and optimization of nanoparticle concentration, temperature, and ball-milling processes of nanoparticles are also carried out using the design of experiment (Doe) approach. The L9 orthogonal array-based full factorial design is used for the best optimization result. The response tables with graphs and optimal results have been obtained through the Taguchi method.