Small-signal models of DC-DC converters are often based on a statespace averaging approach, from which both control-oriented and other frequencydomain characteristics, such as input or output impedance, can be derived. Updating these models when extending the converter by filters or non-trivial loads, or adding control loops, can become a tedious task, however. To simplify this potentially error-prone process, a modular modeling approach is being proposed in this article. It consists of small state-space models for certain building blocks of a converter system on the one hand, and standardized operations for connecting these subsystem models to an overall converter system model on the other hand. The resulting state-space system model builds upon a two-port converter description and allows the extraction of control-oriented and impedance characteristics at any modeling stage, be it open loop or closed loop, single converter or series connections of converters. The ease of creating more complex models enabled by the proposed approach is also demonstrated with examples comprising multiple control loops or cascaded converters.