The mechanisms of serotonin (5-HT)-induced contraction and relaxation were studied in the rat anococcygeus muscle. In the presence of prazosin (1 nM) and ketanserin (10 nM), concentration/response curves to 5-HT were shifted to the right and the maximum effects were not affected (pKB values 9.09+/-0.29 and 8.66+/-0.06 for prazosin and ketanserin, respectively). On contrary, guanethidine (10 microM) antagonised the 5-HT-induced contractions non-competitively. In the presence of guanethidine, prazosin or ketanserin further inhibited the responses to 5-HT at lower concentrations. The serotonergic receptor agonists 5-carboxamidotrypamine, metoclopramide and sumatriptan did not produce any effect in tissues under baseline conditions. 5-HT caused concentration-dependent relaxation in phenylephrine (1 microM) -precontracted preparations in the presence of guanethidine (10 microM). Tetrodotoxin (1 microM), N(G)-nitro-L-arginine (100 microM) and capsaicin (1 microM) were ineffective in antagonising the relaxation induced by 5-HT. The serotonergic receptor antagonists: ketanserin (0.1 microM), ICS 205930 (10 microM), GR 113808 (10 microM), GR 55562 (10 microM), methiothepin (10 nM), mesulergine (10 microM), ritanserin (0.1 microM) and spiperon (0.1 microM) did not antagonise 5-HT-induced relaxation. On the other hand, clozapine (0.01-0.1 microM) and metergoline (0.1-1 microM) attenuated the relaxation induced by 5-HT. These results demonstrate that 5-HT contracts rat anococcygeus muscle directly by an action on alpha1-adrenergic receptors and indirectly by releasing noradrenaline from adrenergic nerve endings. None of the 5-HT receptors plays a role in the 5-HT-induced contractions. Moreover 5-HT induces concentration-dependent relaxation in the precontracted preparations which apparently are not mediated through known 5-HT receptor types.