The inhibitor-of-apoptosis proteins (IAPs) regulate programmed cell death by inhibiting members of the caspase family of enzymes. Recently, a mammalian protein called Smac (also named DIABLO) was identified that binds to the IAPs and promotes caspase activation. Although undefined in the X-ray structure, the amino-terminal residues of Smac are critical for its function. To understand the structural basis for molecular recognition between Smac and the IAPs, we determined the solution structure of the BIR3 domain of X-linked IAP (XIAP) complexed with a functionally active nine-residue peptide derived from the N terminus of Smac. The peptide binds across the third beta-strand of the BIR3 domain in an extended conformation with only the first four residues contacting the protein. The complex is stabilized by four intermolecular hydrogen bonds, an electrostatic interaction involving the N terminus of the peptide, and several hydrophobic interactions. This structural information, along with the binding data from BIR3 and Smac peptide mutants reported here, should aid in the design of small molecules that may be used for the treatment of cancers that overexpress IAPs.

Inhibitor of apoptosis (IAP) proteins are overexpressed in many cancers and have been implicated in tumor growth, pathogenesis, and resistance to chemo- or radiotherapy. On the basis of the NMR structure of a SMAC peptide complexed with the BIR3 domain of X-linked IAP (XIAP), a novel series of XIAP antagonists was discovered. The most potent compounds in this series bind to the baculovirus IAP repeat 3 (BIR3) domain of XIAP with single-digit nanomolar affinity and promote cell death in several human cancer cell lines. In a MDA-MB-231 breast cancer mouse xenograft model, these XIAP antagonists inhibited the growth of tumors. Close structural analogues that showed only weak binding to the XIAP-BIR3 domain were inactive in the cellular assays and showed only marginal in vivo activity. Our results are consistent with a mechanism in which ligands for the BIR3 domain of XIAP induce apoptosis by freeing up caspases. The present study validates the BIR3 domain of XIAP as a target and supports the use of small molecule XIAP antagonists as a potential therapy for cancers that overexpress XIAP.

Overexpression of the antiapototic proteins Bcl-2 and Bcl-xL provides a common mechanism through which cancer cells gain a survival advantage and become resistant to conventional chemotherapy. Inhibition of these prosurvival proteins is an attractive strategy for cancer therapy. We recently described the discovery of a selective Bcl-xL antagonist that potentiates the antitumor activity of chemotherapy and radiation. Here we describe the use of structure-guided design to exploit a deep hydrophobic binding pocket on the surface of these proteins to develop the first dual, subnanomolar inhibitors of Bcl-xL and Bcl-2. This study culminated in the identification of 2, which exhibited EC50 values of 8 nM and 30 nM in Bcl-2 and Bcl-xL dependent cells, respectively. Compound 2 demonstrated single agent efficacy against human follicular lymphoma cell lines that overexpress Bcl-2, and efficacy in a murine xenograft model of lymphoma when given both as a single agent and in combination with etoposide.

Development of a rationally designed potentiator of cancer chemotherapy, via inhibition of Bcl-X(L) function, is described. Lead compounds generated by NMR screening and directed parallel synthesis displayed sub-microM binding but were strongly deactivated in the presence of serum. The dominant component of serum deactivation was identified as domain III of human serum albumin (HSA); NMR solution structures of inhibitors bound to both Bcl-X(L) and HSA domain III indicated two potential optimization sites for separation of affinities. Modifications at both sites resulted in compounds with improved Bcl-X(L) binding and greatly increased activity in the presence of human serum, culminating in 73R, which bound to Bcl-X(L) with a K(i) of 0.8 nM. In a cellular assay 73R reversed the protection afforded by Bcl-X(L) overexpression against cytokine deprivation in FL5.12 cells with an EC(50) of 0.47 microM. 73R showed little effect on the viability of the human non small cell lung cancer cell line A549. However, consistent with the proposed mechanism, 73R potentiated the activity of paclitaxel and UV irradiation in vitro and potentiated the antitumor efficacy of paclitaxel in a mouse xenograft model.