Aortic aneurysms are inherently unpredictable. One can never be sure whether any given aneurysm may rupture or dissect. Clinically, the criteria for surgical intervention are based on size and growth rate, but it remains difficult to identify a high-risk aneurysm, which may require intervention before the cutoff criteria, vs. an aneurysm than can be treated safely by more conservative measures. In this work, we created a computational microstructural model of a medial lamellar unit (MLU) incorporating (1) growth and remodeling laws applied directly to discrete, individual fibers, (2) separate but interacting fiber networks for collagen, elastin, and smooth muscle, (3) active and passive smooth-muscle cell mechanics, and (4) failure mechanics for all three fiber types. The MLU model was then used to study different pathologies and microstructural anomalies that may play a role in vascular growth and failure. Our model recapitulated many aspects of arterial remodeling under hypertension with no underlying genetic syndrome including remodeling dynamics, tissue mechanics, and failure. Syndromic effects (smooth muscle cell dysfunction or elastin fragmentation) drastically changed the simulated remodeling process, tissue behavior, and tissue strength. Different underlying pathologies were able to produce similarly dilatated vessels with different failure properties, providing a partial explanation for the imperfect nature of aneurysm size as a predictor of outcome.