It has been found that a cloud building energy management system (BEMS) alone cannot support increasing numbers of end devices (e.g., energy equipment and IoT devices) and emerging energy services efficiently. To resolve these limitations, this paper proposes Fog BEMS, which applies an emerging fog computing concept to a BEMS. Fog computing places small computing resources (fog nodes) just next to end devices, and these nodes process data in real time and manage local contexts. In this way, the BEMS becomes distributed and scalable. However, existing fog computing models have barely considered scenarios where many end devices and fog nodes are deployed and interconnected. That is, they do not scale up and cannot be applied to scalable applications like BEMS. To solve the problem, this paper (i) designs a fog network where a list of functionally heterogeneous nodes is deployed in a hierarchy for collaboration and (ii) designs an agent-based, modular programming model that eases the development and management of computing services at a fog node. We develop a prototype of a fog node and build a real-world testbed on a campus to demonstrate the feasibility of the proposed system. We also conduct experiments, and results show that Fog BEMS is scalable enough for a node to connect up to 900 devices and that network traffic is reduced by 27.22–97.63%, with varying numbers of end devices.