CERN Computing Seminar

Stork: Making Data Placement a First Class Citizen in the Grid

by Tevfik Kosar (University of Wisconsin-Madison, USA)

Europe/Zurich
IT Auditorium (CERN)

IT Auditorium

CERN

Description
<link rel="stylesheet" type="text/css" href="http://cern.ch/cseminar/CDS/style.css" />

Data placement is an essential part of today's Grid applications. Moving the data close to the application for efficiency, and replicating the data for reliability, are crucial. The increasing data requirements of both scientific and commercial applications, and collaborative access to these data make this problem even more important. In the current approach, data placement is regarded as a side effect of computation. Our goal is to make data placement a first class citizen in the Grid just like the computational jobs. They will be queued, scheduled, monitored, managed, and even check-pointed. Since data placement jobs have different characteristics than computational jobs, they cannot be treated in the exact same way as computational jobs are treated. For this purpose, we are proposing a framework that can be considered as a "data placement subsystem" for the Grid, similar to the I/O subsystem in operating systems. This framework includes a specialized scheduler for data placement, a high level planner aware of data placement jobs, a resource broker/policy enforcer and some optimization tools. The proposed system can perform reliable and efficient data placement, it can recover from all kinds of failures without any human intervention, and it can dynamically adapt to the environment at the execution time.


Organiser(s): Julian Blake / IT Department
More information: http://cern.ch/computing-seminars
© CERN 2005 - Miguel Angel Marquina / IT Department
document
more information
transparencies