San over wan a new way of solving the grid data access bottleneck Dr. Wolfgang Mertz

SAN over WAN - a new way of solving the GRID data access bottleneck

• Dr. Wolfgang Mertz

• Business Development Manager for Storage in EMEA

• [e-mail]

Data Growth Trends

2 Buzzwords in IT Industry

• Server Consolidation

• maybe in a commercial environment
• usually not in a technical environment
• a hammer is a hammer, a screwdriver is a screwdriver
• an HPC system cannot be used as a HPV system

• Storage Consolidation

• DAS -> NAS -> SAN

History of Storage Architectures DAS - Direct Attached Storage

• pro

• appropriate performance

• con

• distributed, expensive administration
• data may not be where it is needed
• multiple copies of data stored

History of Storage Architectures NAS - Network Attached Storage

• pro

• centralized, less expensive administration
• one copy of data
• access from every system

• con

• network performance is the bottleneck

History of Storage Architectures SAN - Storage Area Network

• pro

• centralized administration
• performance equivalent to DAS

• con

• NO FILE SHARING
• multiple copies of data stored

How does that translate to a GRID Environment?

• Storage Consolidation

• useful in a local environment (GRID node)
• does not work between remote GRID nodes

• Current Data Access between GRID Nodes

• Data has to be copied before/after the execution of a job
• Problems
• copy process has to be done manually or included in the job script
• copy can take long
• multiple copies of data
• additional disk space needed
• revision problem

What if...

• ... a SAN would have the same file sharing capability as a NAS?

• ... one could build a SAN between different buildings/sites/cities and not loose performance?

Storage Area Networks (SAN) The High Performance Solution

SGI InfiniteStorage Shared FileSystem (CXFS)

Fibre Channel over SONET/SDH The High Efficiency, Long Distance Alternative

LightSand Solution for building a Global-SAN

LightSand Products

• S-600

• 2 ports FC and/or IP 1Gb/s
• Point-to-point SAN interconnect over SONET/SDH OC-12c (622 Mb/s bandwidth)
• Low latency (approximately 50 µSec)

• S-2500

• 3 ports FC and/or IP 1Gb/s
• Point-to-point SAN interconnect over SONET/SDH OC-48c (2.5 Gb/s bandwidth)
• Point-to-multipoint SAN interconnect over SONET/SDH (up to 5 SAN islands. 622 Mb/s per link)
• Low latency (approximately 50 µSec)

Data Movement Today – A Recent Case Study

• Scientists at LANL currently dump 100GB of supercomputing data to tape and FedEx it to SNL because it is faster than trying to use the existing 155Mb/s IP WAN connection

• Actual measured throughput of 16Mb/s! (10% bandwidth utilization)
• http://www-unix.mcs.anl.gov/discovery/wufeng.htm

The Better Way – Directly Between Storage Systems

• Using LightSand gateways, the same data could be transferred in a few minutes!

What does that mean for a GRID Environment?

• Full Bandwidth Data Access across the GRID

• No Multiple Copies of Data

• avoid the revision problem
• do not waste disk space

• Make GRID Computing more efficient

Highly Integrated, Massively Scalable Systems

SGI InfiniteStorage Product Line