Hot isostatic pressing (HIP) technology can directly produce nearly clean shaped workpieces that meet the requirements while ensuring machining accuracy and surface quality. Usually, people use numerical simulation methods to reduce experimental costs. Generally, a uniform powder relative density distribution of about 65% is used in the simulation. However, in practical engineering, we found that even with additional tools such as vibration tables, the powder filling is not uniform. The non-uniform distribution causes uneven shrinkage of the powder and capsule after HIP. In this paper, a numerical model for HIPing of Ti-6Al-4V powder is developed to improve the prediction by comparing the uniform and non-uniform initial powder distribution. The results show that different initial relative density distributions affect the powder densification process and further affect the deformation of the capsule. It also leads to non-uniform stress distribution after HIP, which increases the risk of capsule rupture. The analysis of the numerical simulation results and the comparison with the experimental results highlights that taking into account the non-uniform powder distribution inside the capsule is vital to improve numerical results and produce near-net shape components. The maximum error of the simulation with the usual initial relative density setting of 65% is 4.2%. However, considering the uneven distribution of initial powder, the maximum error is reduced to 3.16%, and the average error is also less than 2%.