As the technology nodes scale down to 22nm and beyond, Double Patterning Lithography (DPL) has been considered as a practical solution for manufacturing process. Compared with Litho-Etch-Litho-Etch (LELE), Self-Aligned Double Patterning (SADP) has better overlay tolerance. Two types of SADP process are popularly used for the state-of-the-art lithography patterning: Spacer-Is-Dielectric (SID) and Spacer-Is-Metal (SIM). Meanwhile, Self-Aligned Quadruple Patterning (SAQP), as a natural extension of SADP, is expected to be one of the major solutions for future process requirement after the 16nm/14nm technology node. In order to have better decomposability of layout patterns, we consider SIM type SADP/SAQP during detailed routing stage. The idea of color pre-assignment is adopted and a graph model is proposed which greatly simplifies the problem and reduces design rule violation. Then, the negotiated congestion based scheme is applied for detailed routing based on our proposed graph model. Compared with other state-ofart works, our approach does not produce any side overlay error and no design rule violation is reported. Meanwhile, a better solution in terms of total wirelength, via count, routability, and runtime is achieved.