We explore the behavior of richly connected inhibitory neural networks under parameter changes that correspond to weakening of synaptic efficacies between network units, and show that transitions from irregular to periodic dynamics are common in such systems. The weakening of these connections leads to a reduction in the number of units that effectively drive the dynamics and thus to simpler behavior. We hypothesize that the multiple interconnecting loops of the brain's motor circuitry, which involve many inhibitory connections, exhibit such transitions. Normal physiological tremor is irregular while other forms of tremor show more regular oscillations. Tremor in Parkinson's disease, for example, stems from weakened synaptic efficacies of dopaminergic neurons in the nigro-striatal pathway, as in our general model. The multiplicity of structures involved in the production of symptoms in Parkinson's disease and the reversibility of symptoms by pharmacological and surgical manipulation of connection parameters suggest that such a neural network model is appropriate. Furthermore, fixed points that can occur in the network models are suggestive of akinesia in Parkinson's disease. This model is consistent with the view that normal physiological systems can be regulated by robust and richly connected feedback networks with complex dynamics, and that loss of complexity in the feedback structure due to disease leads to more orderly behavior.