Particle physics has an ambitious and broad experimental programme for the coming decades. This programme requires large investments in detector hardware, either to build new facilities and experiments, or to upgrade existing ones. Similarly, it requires commensurate investment in the R&D of software to acquire, manage, process, and analyse the shear amounts of data to be recorded. In planning for the HL-LHC in particular, it is critical that all of the collaborating stakeholders agree on the software goals and priorities, and that the efforts complement each other. In this spirit, this white paper describes the R&D activities required to prepare for this software upgrade.

No abstract

Network traffic optimisation is difficult as the load is by nature dynamic and seemingly unpredictable. However, the increased usage of file transfer services may help the detection of future loads and the prediction of their expected duration. The NOTED project seeks to do exactly this and to dynamically adapt network topology to deliver improved bandwidth for users of such services. This article introduces, and explains the features of, the two main components of NOTED, the Transfer Broker and the Network Intelligence component. The Transfer Broker analyses all queued and on-going FTS transfers, producing a traffic report which can be used by network controllers. Based on this report and its knowledge of the network topology and routing, the Network Intelligence (NI) component makes decisions as to when a network reconfiguration could be beneficial. Any Software Defined Network controller can then apply these decision to the network, so optimising transfer execution time and reducing operating costs.

No abstract

We present NOTED, a project to better exploit WAN bandwidth for Rucio and FTS data transfers. We describe the developed software tool-box and a successful demonstration of automated load-balancing in the production network. The first component is the Transfer Broker: It identifies large data transfers for which network reconfiguration is both possible and beneficial, and translates the transfer information into parameters that can be used by network controllers. It will make this information available via a public interface. The second component is a demonstration of a Network Controller. It periodically reads the parameters provided by the Transfer Broker, and so decides which actions to apply to improve the network utilisation for a given transfer.