Classical approximate dynamic programming techniques based on state-space gridding become computationally impracticable for high-dimensional problems. Policy search techniques cope with this curse of dimensionality issue by searching for the optimal control policy in a restricted parameterized policy space. We here focus on the case of discrete action space and introduce a novel policy parametrization that adopts particles to describe the map from the state space to the action space, each particle representing a region of the state space that is mapped into a certain action. The locations and actions associated with the particles describing a policy can be tuned by means of a recently introduced policy gradient method with parameter-based exploration. The task of selecting an appropriately sized set of particles is here solved through an iterative policy building scheme that adds new particles to improve the policy performance and is also capable of removing redundant particles. Experiments demonstrate the scalability of the proposed approach as the dimensionality of the state-space grows.

District heating systems have an important role in increasing the efficiency of the heating and cooling sector, especially when coupled to combined heat and power plants. However, in the transition towards decarbonization, current systems show some challenges for the integration of Renewable Energy Sources and Waste Heat. In particular, a crucial aspect is represented by the operating temperatures of the network. This paper analyzes two different approaches for the decrease of operation temperatures of existing networks, which are often supplying old buildings with a low degree of insulation. A simulation model was applied to some case studies to evaluate how a low-temperature operation of an existing district heating system performs compared to the standard operation, by considering two different approaches: (1) a different control strategy involving nighttime operation to avoid the morning peak demand; and (2) the partial insulation of the buildings to decrease operation temperatures without the need of modifying the heating system of the users. Different temperatures were considered to evaluate a threshold based on the characteristics of the buildings supplied by the network. The results highlight an interesting potential for optimization of existing systems by tuning the control strategies and performing some energy efficiency operation. The network temperature can be decreased with a continuous operation of the system, or with energy efficiency intervention in buildings, and distributed heat pumps used as integration could provide significant advantages. Each solution has its own limitations and critical parameters, which are discussed in detail.

No abstract

No abstract

No abstract