May 18 – 19, 2017
University of Michigan
America/Detroit timezone

Data Ingestion at Scale

May 19, 2017, 9:30 AM
30m
North Quad room 2435 (University of Michigan)

North Quad room 2435

University of Michigan

School of Information 105 S. State St. Ann Arbor, MI 48109-1285
Presentation Complementary Technology Solutions Complementary Technology Solutions

Speaker

Jeffrey Sica (University of Michigan)

Description

HPC traditionally handles data at rest. The acquisition of streaming data presents a different set of challenges that, at scale, can be difficult to tackle. The approach to building data ingestion infrastructure at ARC-TS involves treating every service as a swappable building block. With this pluggable design using Docker containers you are free to choose which component is best. We will use an example use case to show how data is being generated, ingested, and how each component in the stack can be replaced.

Primary author

Jeffrey Sica (University of Michigan)

Presentation materials