Communication plays a pivotal role in distributed optimization problems, where unavoidable communication delays are presented. This research studies the distributed constrained optimization problem concerning second‐order multi‐agent systems with double‐integrator under time‐varying communication delays. An adaptive distributed optimization algorithm using multi‐agent system consensus technique and Karush–Kuhn–Tucker conditions is developed to deal with this problem. The local constraint term is solved adaptively through local dual Lagrange multipliers. When the cost function is strongly convex, and the communication topology is undirected and connected, we employ the Lasalle invariance principle to analyze the delay effects on convergence analysis. Moreover, we give an upper bound on communication delay. Finally, the provided numerical simulation examples demonstrate that the developed method is robust for the limited communication delay and the derived results are conservative.