Ketanserin is the prototypic 5-HT2 serotonin antagonist; although it has been an important tool for the study of serotonin pharmacology, it has had relatively little impact on drug design because remarkably little is known about its structure-affinity relationships. Furthermore, ketanserin also binds at 5-HT1C receptors and even less is known about the influence of its structural features on 5-HT1C receptor affinity. The present study reveals that the fluoro and carbonyl groups of the 4-fluorobenzoyl portion of ketanserin make small contributions to 5-HT2 binding and that the intact benzoylpiperidine moiety is an important feature. Ring-opening of the piperidine ring reduces affinity. Although the quinazoline-2,4-dione moiety also contributes to binding, it appears to play a smaller role and can be structurally simplified with retention of 5-HT2 affinity. N-(4-Phenylbutyl)-4-(4-fluorobenzoyl)piperidine (39), for example, binds with nearly the same affinity (Ki = 5.3 nM) as ketanserin (Ki = 3.5 nM). All of the compounds examined bind at 5-HT1C sites with lower affinity than ketanserin, and some of the simplified analogues bind with nearly 10 times the 5-HT2 versus 5-HT1C selectivity of ketanserin; however, none displays > 120-fold selectivity. Several of the compounds, such as the amide 32 and the urea 33 represent examples of new structural classes of 5-HT2 ligands.

Arylpiperazines are nonselective agents that bind at 5-HT3 serotonin receptors with moderate to high affinity, whereas 1-phenylbiguanide is a low-affinity but more selective 5-HT3 agonist. In an attempt to enhance the affinity of the latter agent, and working with the assumption that similarities might exist between the binding of the two types of agents, we formulated structure-activity relationships for the binding of the arylpiperazines and then incorporated those substituents, leading to high affinity for the arylpiperazines, into 1-phenylbiguanide. A subsequent investigation examined the structure-activity relationships of the arylbiguanides and identified arylguanidines as a novel class of 5-HT3 ligands. Although curious similarities exist between the structure-activity relationships of the arylpiperazines, arylbiguanides, and arylguanidines, it cannot be concluded that all three series of compounds are binding in the same manner. Furthermore, upon investigating pairs of compounds in the three series, the arylpiperazines behaved as 5-HT3 antagonists (von Bezold-Jarisch assay) whereas the arylbiguanides and arylguanidines acted as 5-HT3 agonists.

DOM [i.e., 1-(2,5-dimethoxy-4-methylphenyl)-2-aminopropane] is a 5-HT1C/2 serotonin agonist that exerts stimulus control of behavior in animals. In order to determine if the discriminative stimulus effect of DOM is 5-HT1C- or 5-HT2-mediated, it would be informative to conduct tests of stimulus antagonism with a 5-HT1C- or 5-HT2-selective antagonist. To date, no such agents exist. Although the neuroleptic agent spiperone binds at D2 dopamine receptors and 5-HT1A serotonin receptors, (a) it displays about a 1000-fold selectivity for 5-HT2 versus 5-HT1C sites and (b) it has been used as a "5-HT2-selective" antagonist. Because spiperone is a behaviorally disruptive agent, it is not suitable for use in drug-discrimination studies. Using the spiperone molecule as a starting point, a limited structure-affinity investigation was conducted in order to identify a suitable antagonist with high affinity and selectivity for 5-HT2 receptors, and yet an antagonist that might lack the disruptive actions of spiperone. Various modifications of the spiperone molecule were examined, but most resulted in decreased 5-HT2 affinity or in loss of selectivity. One compound, 8-[3-(4-fluorophenoxy)propyl]-1-phenyl-1,3,8-triazaspiro[4.5]de can-4-on e (26), was shown to bind at 5-HT2 sites with high affinity (Ki = 2 nM) and > 2,000-fold selectivity versus 5-HT1C sites. In tests of stimulus antagonism using rats trained to discriminate 1 mg/kg of DOM from saline vehicle, 26 behaved as a potent antagonist (ED50 = 0.003 mg/kg) and lacked the disruptive effects associated with spiperone. As such, (a) it would appear that the DOM stimulus is primarily a 5-HT2-mediated, and not 5-HT1C-mediated, phenomenon, and (b) compound 26 may find application in other pharmacologic investigations where spiperone may not be a suitable antagonist.