Antibody–drug conjugates (ADCs) have developed rapidly in recent decades. However, it is complicated to map out a perfect ADC that requires optimization of multiple parameters including antigens, antibodies, linkers, payloads, and the payload-linker linkage. The therapeutic targets of the ADCs are expected to express only on the surface of the corresponding target tumor cells. On the contrary, many antigens usually express on normal tissues to some extent, which could disturb the specificity of ADCs and limit their clinical application, not to mention the antibody is also difficult to choose. It requires to not only target and have affinity with the corresponding antigen, but it also needs to have a linkage site with the linker to load the payloads. In addition, the linker and payload are indispensable in the efficacy of ADCs. The linker is required to stabilize the ADC in the circulatory system and is brittle to release free payload while the antibody combines with antigen. Also, it is a premise that the dose of ADCs will not kill normal tissues and the released payloads are able to fulfill the killing potency in tumor cells at the same time. In this review, we mainly focus on the latest development of key factors affecting ADCs progress, including the selection of antibodies and antigens, the optimization of payload, the modification of linker, payload-linker linkage, and some other relevant parameters of ADCs.