High-performance Data Intensive Distributed Computing

by Brian TIERNEY

Thursday 23 Sept 1999, 10:30 → 14:00 Europe/Zurich

IT Auditorium (CERN)

The Data Intensive Distributed Computing Group at LBNL has developed some distributing computing technologies that we feel could be extremely useful to the HENP community. The first is the Distributed Parallel Storage System (DPSS), which provides cost effective parallel access to many disks, and is tuned for wide-area network access. We have demonstrated a HENP data analysis application with read and write speeds of up to 57 MBytes/sec over a OC-12 WAN using a 4 server DPSS system. Using a DPSS, an application can access remote data even faster than it can access data from a local disk, eliminating the need to copy data sets from the remote site before processing or visualising it. For more information, see: http://www-didc.lbl.gov/DPSS/ The second is The NetLogger Toolkit, which includes tools for instrumenting applications, tools for monitoring hosts and networks, and tools for collecting and visualising monitoring data. Using these tools enables us to do end-to-end monitoring, and detailed performance analysis of every component in the system. This type of analysis is critical for obtaining performance in widely distributed systems. For more information, see: http://www-didc.lbl.gov/NetLogger/

Organiser(s): G.FOLGER / IT