Design of Butler matrices dedicated to Internet of Things and 5th generation (5G) mobile systems—where small size and high performance are of primary concern—is a challenging task that often exceeds capabilities of conventional techniques. Lack of appropriate, unified design approaches is a serious bottleneck for the development of Butler structures for contemporary applications. In this work, a low-cost bottom-up procedure for rigorous and unattended design of miniaturized 4 × 4 Butler matrices is proposed. The presented approach exploits numerical algorithms (governed by a set of suitable objective functions) to control synthesis, implementation, optimization, and fine-tuning of the structure and its individual building blocks. The framework is demonstrated using two miniaturized matrices with nonstandard output-port phase differences. Numerical results indicate that the computational cost of the design process using the presented framework is over 80% lower compared to the conventional approach. The footprints of optimized matrices are only 696 and 767 mm2, respectively. Small size and operation frequency of around 2.6 GHz make the circuits of potential use for mobile devices dedicated to work within a sub-6 GHz 5G spectrum. Both structures have been benchmarked against the state-of-the-art designs from the literature in terms of performance and size. Measurements of the fabricated Butler matrix prototype are also provided.