Androgen receptor (AR) is closely associated with a group of hormone‐related diseases including the cancers of prostate, breast, ovary, pancreas, etc and anabolic deficiencies such as muscle atrophy and osteoporosis. Depending on the specific type and stage of the diseases, AR ligands including not only antagonists but also agonists and modulators are considered as potential therapeutics, which makes AR an extremely interesting drug target. Here, we at first review the current understandings on the structural characteristics of AR, and then address why and how AR is investigated as a drug target for the relevant diseases and summarize the representative antagonists and agonists targeting five prospective small molecule binding sites at AR, including ligand‐binding pocket, activation function‐2 site, binding function‐3 site, DNA‐binding domain, and N‐terminal domain, providing recent insights from a target and drug development view. Further comprehensive studies on AR and AR ligands would bring fruitful information and push the therapy of AR relevant diseases forward.