We propose a new framework for topology optimization based on the boundary element discretization and kernel-independent fast multipole method (KIFMM). The boundary value problem for the considered partial differential equation is reformulated as a surface integral equation and is solved on the domain boundary. Volume solution at selected points is found via surface integrals. At every iteration of the optimization process, the new boundary is extracted as a level set of a topological derivative. Both surface and volume solutions are accelerated using KIFMM. The obtained technique is highly universal, fully parallelized, it allows achieving asymptotically the best performance with the optimization iteration complexity proportional to a number of surface discretization elements. Moreover, our approach is free of the artifacts that are inherent for finite element optimization techniques, such as "checkerboard" instability. The performance of the approach is showcased on few illustrative examples.