We study the behavior of center sectors in pure SU(4) lattice gauge theory at finite temperature. The center sectors are defined as spatial clusters of neighboring sites with values of their local Polyakov loops near the same center elements. We study the connectedness and percolation properties of the center clusters across the deconfinement transition. We show that for SU(4) gauge theory deconfinement cannot be described as a percolation transition of center clusters, a finding which is different from pure SU(2) or pure SU(3) Yang Mills theory, where the percolation description even allows for a continuum limit.