In the optimal design of modular structures and mechanisms, two key questions must be answered: (i) what should the topology of individual modules be like and (ii) how should modules be arranged at the product scale? We address these challenges by proposing a bi-level sequential strategy that combines free material design, clustering techniques, and topology optimization. First, using free material optimization enhanced with post-processing for checkerboard suppression, we determine the distribution of elasticity tensors at the product scale. To extract the sought-after modular arrangement, we partition the obtained elasticity tensors with a novel deterministic clustering algorithm and interpret its outputs within Wang tiling formalism. Finally, we design interiors of individual modules by solving a single-scale topology optimization problem with the design space reduced by modular mapping, conveniently starting from an initial guess provided by free material optimization. We illustrate these developments with three benchmarks first, covering compliance minimization of modular structures, and, for the first time, the design of compliant modular mechanisms. Furthermore, we design a set of modules reusable in an inverter and in gripper mechanisms, which ultimately pave the way towards the rational design of modular architectured (meta)materials.