Coordinated multi-point (CoMP) is a promising technique in next generation cellular networks. Compared with traditional mobile networks, one of the important design problems in CoMP is clustering, which decides how the base stations cooperate with each other. Channel state information (CSI) is needed in clustering decisions in CoMP. Most previous works assume that perfect CSI is available. However, practical systems suffer from constraints imposed by backhaul networks, which are used for CSI exchange. In this paper, we study the clustering and rate allocation problem in CoMP with delayed CSI. We present a decision theoretic approach to this problem. Specifically, we model such a system in the framework of networked Markov decision process (networked-MDP) with delays, which is equivalent to a partial observable Markov decision process (POMDP). We derive an optimal policy for such POMDP with low computation complexity. Simulation results are provided to show promising gain achieved in the proposed scheme over existing schemes especially when the delay is large and the channel coherence time is small.