We present CosmoHub, a web platform to perform interactive analysis of massive cosmological data without any SQL knowledge. CosmoHub is built on top of Apache Hive, which is an Apache Hadoop ecosystem component, which facilitates reading, writing, and managing large datasets.
CosmoHub is hosted at the Port de Informació Científica (PIC) and currently provides support to several international...
HammerCloud is a testing service to run continuous tests or on-demand large-scale stress tests of the WLCG resources with real-life experiment jobs.
HammerCloud is used by the ATLAS and CMS experiments in production. It has been a useful service to commission both compute resources and various components of the complex distributed systems of the LHC experiments, as well as integral part of...
Brookhaven Lab recently acquired an Intel Knight's Landing (KNL) cluster consisting of 144 nodes connected with a dual-rail OmniPath (OPA) fabric. We will detail our experiences integrating this cluster into our environment, testing the performance and deugging issues relating to the fabric and hardware. Details about the integration with the batch system (Slurm) and performance issues found...
Update to JLab' Fall 2016 SciPhi-XVI KNL talk, to include the addition of 64 nodes into our Knights Landing cluster, reaching #397 on the Top500 list at 429.5 TFlops and #10 on the Green500 list at 3836.6 MFLOPS/W. It will include an overview of our cluster configuration updates, Omni-Path fabric, benchmarking, integration with Lustre and NFS over Infiniband, as well as current open issues.
An update of the activity of the HEPiX Benchmarking Working Group will be reported.
IHEP cluster with more 10,000 jobs slots was migrated from PBS to HTCondor by the end of 2016. The report talks about the the sharing pool scheduling policy deployed at IHEP cluster to promote the resource utility. And the experience of HTCondor management would be talked in the report.
The multi-user pilot job (MUPJ) model has become deeply embedded in the LHC computing ecosystem. In this model, the pilot job sent to a site batch will dynamically pull down one or more user payload jobs as it is running at the site.
While providing the experiments with utmost flexibility, the MUPJ presents challenges in isolation (preventing payloads from interacting with the pilot) and...
The HEPiX Benchmarking Working group has been investigating fast benchmark applications with the objective of identifying candidates that can run quickly enough to avoid wasting compute resources, but still capable of accurately representing HEP workloads. Understanding how the underlying processor microarchitecture affects the results of these benchmarks is important to foresee scenarios...