Lambda Station: On-Demand Flow Based Routing for Data Intensive Grid Applications Over Multitopology Networks

Abstract

Lambda Station is an ongoing project of Fermi National Accelerator Laboratory and the California Institute of Technology. The goal of this project is to design, develop and deploy network services for path selection, admission control and flow based forwarding of traffic among dataintensive Grid applications such as are used in High Energy Physics and other communities. Lambda Station deals with the last-mile problem in local area networks, connecting production clusters through a rich array of wide area networks. Selective forwarding of traffic is controlled dynamically at the demand of applications. This paper introduces the motivation of this project, design principles and current status. Integration of Lambda Station client API with the essential Grid middleware such as the dCache/SRM Storage Resource Manager is also described. Finally, the results of applying Lambda Station services to development and production clusters at Fermilab and Caltech over advanced networks such as DOE's UltraScience Net and NSF's UltraLight is covered. PROJECT OVERVIEWThe main goal of Lambda Station project is to design, develop and deploy a network path selection service to interface production storage and computing facilities with advanced research networks. In the future, when corresponding API are available Lambda Station will also take on the task of negotiating with reservation or provisioning systems that may regulate the WAN control planes.Policy based routing (PBR) is used to implement flowspecific routing in the LAN and at the border between LAN and WAN. In the next section of this paper we will discuss how Lambda Station serves the unprecedented demands for data movement by running experiments such as CDF, D0, and BaBar as well as upcoming LHC experiments. From our point of view, available data communication technology will not be able to satisfy these demands simply by increasing bandwidth in LANs and commodity WANs due to technology limitations and high deployment and operational costs. Selective forwarding on per flow basis to alternate network paths is desirable for high impact data while leaving other traffic on regular paths. The ability to selectively forward traffic requires developing a control unit that is able to dynamically reconfigure forwarding of specific flows within local production-use routers on demand of applications. We refer to such a control unit as Lambda Station. If one envisions the optical network paths provided by advanced optical-based research networks as high bandwidth data railways, then Lambda Station would functionally be the railroad terminal that regulates which flows at the local site get directed onto the high bandwidth data railways. Lambda Station coordinates network path availability, scheduling, and setup, directs appropriate forwarding within the local network infrastructure, and provides the application with the necessary information to utilize the high bandwidth path. Having created Lambda Station, we introduce awareness and exploitation of advanced networking into d...

A User Driven Dynamic Circuit Network Implementation

“…TeraPaths [21] and LambdaStation [22] are two projects designed to dynamically manage site LAN resources. In addition to coordinating resource allocations between the site gateway and the end-host, they communicate with OSCARS and/or Internet2's IDC via clients using the WSDL description to create host-to-host VCs.…”

MNEMONIC: A Network Environment for Automatic Optimization and Tuning of Data Movement over Advanced Networks

“…Other clearly related work involves dynamic network resource allocation. Systems like Terapaths [11] and LambdaStation [4] and VINCI [31] are all approaches related to the DCN/SDN approach, which uses the OSCARS system [24]. These groups are collaborating and as their approaches move closer together, we will be able to take advantage of their advances.…”

Improving the bulk data transfer experience

“…There is neither the assurance that a packet will be delivered to its destination, nor any transport predictability (such as latency and jitter) when the packet is in transit. The requirements for user driven bandwidth allocation has spawned several activities such as the DOE funded Lambda Station (Bobyshev et al, 2006), On-Demand Secure Circuits and Advance Reservation System (OSCARS) (Guok et al 2006), TeraPaths (Bradley et al, 2006) and UltraScience Net (Rao et al, 2005) projects, the National Science Foundation (NSF) (National Science Foundation, 2007) funded Circuit-switched Highspeed End-to-End Transport Architecture (CHEETAH) (Veeraraghavan et al, 2003) and Dynamic Resource Allocation via GMPLS Optical Networks (DRAGON) (Yang et al, 2006) projects, Internet2's (Internet2, 2007) Bandwidth Reservation for User Work (BRUW) (Riddle, 2005) and Hybrid Optical and Packet Infrastructure (HOPI) (Boyles, 2004) projects, CANARIE's (CANARIE Inc., 2007) User-controlled Lightpath (UCLP) project, and GÉANT's (GÉANT, 2007) AUTOBahn (Sevasti, 2006) and Advanced Multi-domain Provisioning System (AMPS) (Patil, 2006) activities.…”