Drones have a wide range of applications in urban environments as they can both enhance people's daily activities and commercial activities through various operations and deployments. With the increasing number of drones, flight safety and efficiency become the main concern, and effective drone operations can make a difference. Accordingly, 4D path planning for drone operations is the focus of this paper, and the swarm-based method is proposed to solve this complicated optimization problem. Under the framework of 'AirMatrix', the problem is solved in two levels, i.e., 3D path planning for a single drone and conflict resolution among drones. In the multipath planning level, multiple alternative flight paths for each drone are generated to increase the acceptance rate of a flight request. The constraints on a single flight path and two different flight paths are considered. The goal is to obtain several different short flight paths as alternatives. A clustering improved ant colony optimization (CIACO) algorithm is employed to solve the multi-path planning problem. The crowding mechanism is used in clustering, and some improvements are made to strengthen the global and local search ability in the early and later phases of iterations. In the task scheduling level, the conflicts between two drones are defined in two circumstances. One is for the time interval of passing the same path point, another one is for the right-angle collision between two drones. A three-layer fitness function is proposed to maximize the number of permitted flights according to the safety requirement, in which the airspace utilization and the operators' requests are both considered. A 'cross-off' strategy is developed to calculate the fitness value, and a 'distributed-centralized' strategy is applied considering the task priorities of drones. A genetic algorithm (GA)-based task scheduling algorithm is also developed according to the characteristic of the established model. Simulation results demonstrate that 4D flight path of each drone can be generated by the proposed swarmed-based algorithms, and safe and efficient drone operations in a specific airspace can be ensured.