Large Hadron Collider: UltraLight Data Analysis Tools 2007

UltraLight is a collaboration of experimental physicists and network engineers whose purpose is to provide the network advances required to enable petabyte-scale analysis of globally distributed data. Current Grid-based infrastructures provide massive computing and storage resources, but are currently limited by their treatment of the network as an external, passive, and largely unmanaged resource.

On April 24, 2007, UltraLight received the 2007 Internet2 Driving Exemplary Applications (IDEA) Awards program, which recognizes leading innovators who have created and deployed advanced network applications that have enabled transformational progress in research, teaching and learning. UltraLight provides the foundation and services for linking thousands of physicists and scientists around the world who together are investigating the origins of the universe. At SC07, an international team of physicists, computer scientists, and network engineers led by Caltech, University of Michigan, the National Institute of Information Technology in Pakistan, Polytehnica University in Romania, Fermilab, Brookhaven National Lab, and CERN, and partners from Brazil (Rio de Janeiro State University, UERJ, and two of the State Universities of São Paulo, USP and UNESP) and Korea (Kyungpook National University, KISTI) set new records for sustained data transfer among storage systems.

The team demonstrated storage-to-storage data transfer over wide-area networks from a single rack of servers on the exhibit floor. The team’s demonstration of “High Speed Data Distribution for Physics Discoveries at the Large Hadron Collider” achieved a bidirectional peak throughput of 88Gbps and a sustained data flow of more than 80Gbps for two hours among clusters of servers on the show floor and at Caltech, Michigan, Fermilab, CERN, Brazil, Korea, and locations in the US LHCNet network in Chicago, New York, and Amsterdam. On November 14, 2007, Caltech reached 70Gbps of disk-to-disk throughput using a single rack of server equipment, running Caltech’s Fast Data Transfer with a kernel containing the Caltech FAST TCP patch. Caltech reached a peak in one direction of 41Gbps.




Stanford Linear Accelerator Center
Fermi National Accelerator Laboratory
Brookhaven National Laboratory
University of Florida
University of Michigan
Vanderbilt University
Cisco Systems

Korea Institute of Science and Technology Information (KISTI)
Kyungpook National University

University of Manchester

Universidade do Estado do Rio de Janeiro (UERJ)
Universidade Estadual Paulista (UNESP)
Universidade de São Paulo (USP)

National Institute of Information Technology

Polytechnica University