Login

Symposium of the Center for Network and Storage Enabled Collaborative Computational Science

18–19 May 2017
University of Michigan
America/Detroit timezone

Symposium Support / Questions / Problems

Session

Complementary Technology Solutions

19 May 2017, 09:30
North Quad room 2435 (University of Michigan)

North Quad room 2435

University of Michigan

School of Information 105 S. State St. Ann Arbor, MI 48109-1285

Conveners

Complementary Technology Solutions: Session 1

  • Sharon Broude Geva (University of Michigan)

Complementary Technology Solutions: Session 2

  • Sharon Broude Geva (University of Michigan)

Description

User experiences, challenges and requests

Presentation materials

There are no materials yet.

  1. 16. Data Ingestion at Scale
    Jeffrey Sica (University of Michigan)
    19/05/2017, 09:30
    Complementary Technology Solutions
    Presentation

    HPC traditionally handles data at rest. The acquisition of streaming data presents a different set of challenges that, at scale, can be difficult to tackle. The approach to building data ingestion infrastructure at ARC-TS involves treating every service as a swappable building block. With this pluggable design using Docker containers you are free to choose which component is best. We will use...

    Go to contribution page
  2. 17. Bootstrapping Big Data with Spark SQL and Data Frames
    Brock Palen
    19/05/2017, 10:30
    Complementary Technology Solutions
    Presentation

    Apache Spark, a popular open source big data tool form the Hadoop ecosystem is seeing rapid adoption across industry and academia, yet it is still generally not well known. For this talk we will demonstrate some large scale samples of how easy it is to benefit form spark SQL and Data Frames for Python and R programmers.

    Go to contribution page
  3. 18. Highly scalable metadata management with signac
    Carl S. Adorf (University of Michigan)
    19/05/2017, 11:00
    Complementary Technology Solutions
    Presentation

    Continually increasing computational resources and improved efficiency of parallelized software for data generation and manipulation in the field of scientific computation have led to the requirement of more systematic approaches for data management. We present a data management framework designed to work on both desktop computers and in high-performance computing environments with special...

    Go to contribution page
Building timetable...