Governments often supplement the regular military with paramilitaries and pro-government militias (PGMs). However, it is unclear what determines states' selection of these auxiliary forces, and our understanding of how auxiliary force structures develop remains limited. The crucial difference between the two auxiliary types is their embeddedness in official structures. Paramilitaries are organized under the government to support/replace the regular military, while PGMs exist outside the state apparatus. Within a principal-agent framework, we argue that a state's investment in a particular auxiliary force structure is shaped by available resources and capacity, accountability/deniability, and domestic threats. Our results based on quantitative analysis in 1981-2007 find that (1) state capacity is crucial for sustaining paramilitaries, but not PGMs, (2) PGMs, unlike paramilitaries, are more common in states involved in civil conflict, and (3) although both paramilitaries and PGMs are associated with regime instability, there is no significant difference between them in that context.