Assays of behavior in model organisms play an important role in genetic screens, drug testing, and the elucidation of gene-behavior relationships. We have developed an automated, high-throughput imaging and analysis method for assaying behaviors of the nematode C. elegans. We use high-resolution optical imaging to longitudinally record the behaviors of 96 animals at a time in multi-well plates, and computer vision software to quantify the animals' locomotor activity, behavioral states, and egg laying events. To demonstrate the capabilities of our system we used it to examine the role of serotonin in C. elegans behavior. We found that egg-laying events are preceded by a period of reduced locomotion, and that this decline in movement requires serotonin signaling. In addition, we identified novel roles of serotonin receptors SER-1 and SER-7 in regulating the effects of serotonin on egg laying across roaming, dwelling, and quiescent locomotor states. Our system will be useful for performing genetic or chemical screens for modulators of behavior.