News from CERN since the previous HEPiX workshop.
News and updates from IHEP since the last HEPiX Workshop. In this talk we would like to present the status of IHEP site including computing farm, HPC, Grid, data storage ,network and so on.
KEK Central Computer System (KEKCC) has been upgraded by the full-scale hardware replacement and started the operation in September 2020. In this report, we present the specifications of the new KEKCC with the usage achievement of the previous system last year.
The talk will meld the experience of the HSF Workshop, PyHEP and ICHEP and discuss what proved to work best and things that didn’t work as well as hoped.
The KEK Computing Research Center provides various IT services, such as email, WiFi network, data analysis system, etc. A user submits paper application forms to apply those services. Moreover, some services require endorsement by a KEK staff. From the Computing Research Center point of view, we receive applications from many users every day. These forms are checked, processed, and handed out...
The number of BYOD devices at CERN is growing and there is interest in moving from a centrally-managed model to a distributed model where users are responsible for their own devices. Following this strategy, new tools are needed to distribute and - in case of licensed software - also manage licences for applications provided by CERN. Available open source and commercial solutions were...
This is the PIC report for HEPiX Autumn 2020 Workshop
Diamond Light Source is a synchrotron light source in Oxfordshire, UK. We will present the developments at Diamond since HEPiX Autumn 2019 and our plans for the future.
News from the center since Autumn 2019
A Research Digital Management (RDM) repository is a web based service that would provide BNL’s scientific community a means to share and preserve their scientific results while making them Findable, Accessible, Interoperable, and Reusable (FAIR). Towards this goal Invenio, an open-source software framework for building large-scale digital repositories, is being used to create a research data...
The Scientific Data & Computing Center (SDCC) at BNL is migrating its web
content management system from Plone to Drupal. This presentation provides
a status update on the project. Several technologies were evaluated and tested according to facility and user requirements and specifications.
The DBOD Service is a Database as a Service platform that provides MySQL, PostgreSQL, and InfluxDB database instances to CERN users.
During the last few years, more and more critical services have moved to open source database solutions and are now making use of the DBOD service. As a consequence, high availability is expected for some of these services.
This presentation describes how...
This presentation covers the BNLBox file sharing service recently introduced at the Scientific Data & Computing Center (SDCC).
This presentation aims to give an update on the global security landscape from the past year.
The COVID-19 pandemic has introduced a novel challenge for security teams everywhere by expanding the attack surface to include everyone's personal devices / home networks and causing a shift to new, risky software for a remote-first working environment. It was also a chance for attackers to get...
Since years, e-mail is one of the main attack vectors that organisations and individuals face. Malicious actors use e-mail messages to run phishing attacks, to distribute malware, and to send around various types of scams. While technical solutions exist to filter out most of such messages, no mechanism can guarantee 100% efficiency. E-mail recipients themselves are the next, crucial layer of...
This presentation delivers a holistic view of the current and future state of the DNS Load Balancing service at CERN. CERN runs this service to provide the necessary tools for managing the nodes that aliases should present. The service contains three main components:
1.An administrative interface, where users can define aliases and their policies.
2.A client that runs on the nodes to assess...
NOTED is a project that aims to better exploit WAN bandwidth needed by FTS data transfers.
The main component is the Transfer Broker, which interpreters information coming from FTS to identify large data transfer which could benefit of network optimization. The Transfer Broker then enriches the FTS transfers with network information coming from CRIC, the resource database used by ATLAS and...
High Energy Photon Source (HEPS) is the first national high-energy synchrotron radiation light source in Beijing China, and will be ready for users and scientists in 2025. According to the estimated data rates, we predict 30 PB raw experimental data will be produced per month from 14 beamlines at the first stage of HEPS, and the data volume will be even greater after over 90 beamlines are...
While there are a handful of International Gravitational-Wave
Observatory Network (IGWN) submit nodes deployed in the US for LIGO,
Virgo, Kagra (aka IGWN) data-processing pipelines, Nikhef has worked
with LIGO and Virgo collaborators to design a submit node that can be
deployed at EU sites. More EU-based submit nodes will allow additional
points of entry for IGWN computing resources and...
Our HTC cluster using HTCondor has been set up at Bonn University in 2017/2018.
All infrastructure is fully puppetised, including the HTCondor configuration.
OS updates are fully automated, and necessary reboots for security patches are scheduled in a staggered fashion backfilling all draining nodes with short jobs to maximize throughput.
Additionally, draining can also be scheduled for...
High Throughput Computing (HTC) datacenters are a cornerstone of scientific discoveries in the fields of High Energy Physics and Astroparticles Physics. These datacenters provide thousands of users from dozens scientific collaborations with tens of thousands computing cores and Petabytes of storage. The scheduling algorithm used in such datacenters to handle the millions of (mostly...
The HEPiX benchmarking working group has been very active in the past months and will present a report of the activities, concentrating on the new developments after the last report. The new candidate benchmark, designed to overcome the HEP-SPEC06 problems is being published. The HEP Benchmarking Suite has been complemented by GPU and Root benchmark.
The "European" HTCondor workshop was held from 21 to 25 September as a purely virtual event. We will report on topics covered and observations made.
WLCG relies on the network as a critical part of its infrastructure and therefore needs to guarantee effective network usage and prompt detection and resolution of any network issues, including connection failures, congestion and traffic routing. The OSG Networking Area is a partner of the WLCG effort and is focused on being the primary source of networking information for its partners and...
As the scale and complexity of the current HEP network grows rapidly, new technologies and platforms are being introduced that greatly extend the capabilities of today’s networks. With many of these technologies becoming available, it’s important to understand how we can design, test and develop systems that could enter existing production workflows while at the same time changing something as...
The transition of WLCG central and storage services to dual-stack IPv4/IPv6 has gone well, thus enabling the use of IPv6-only CPU resources as agreed by the WLCG Management Board. More and more WLCG data transfers now take place over IPv6. The dual-stack deployment does however result in a networking environment which is much more complex than when using just IPv6. During the last year the...
We will report on recent activities on integrating Federated Identity Management at the Scientific Data & Computing Center (SDCC).
In this contribution we report on the ongoing R&D activity aiming at preparing the EOS ALICE O2 storage cluster for the very demanding requirements of Run 3. After the planned upgrades of LHC and ALICE detectors, the ALICE experiment is expected to increase the data-taking rate handled by the online system and then recorded into permanent storage by one order of magnitude. In order to...
Distributed computing involves processing data from far remote site. We
explore a new type of cache, XCache, developed within XROOTD, and
explore the its capability to improve the access of the jobs to remote
storages. The proof-of-concept is realized within the ESCAPE european
project and measurement of the performances are done using remote sites.
The presentation will develop the...
The Cern VM File System (CVMFS) is a service for fast and reliable software distribution on a global scale. It is capable of delivering scientific software onto physical nodes, virtual machines, and HPC clusters by providing POSIX read-only file system access. Files and metadata are downloaded on-demand by means of HTTP requests and take advantage of aggressive caching on intermediate caches...
The File Transfer Service (FTS) is a fundamental component for the LHC experiments, distributing the majority of the LHC data across the WLCG infrastructure. Tightly integrated with experiment frameworks, it has transferred more than 1 billion files and a total of 950 petabytes of data in 2019 alone. With more than 30 experiments using FTS at CERN and outside, it has steadily gained popularity...
The German Helmholtz Association (HGF) encompasses 19 research institutes distributed all over Germany, covering a wide variety of research topics ranging from particle and material physics over cancer research to marine biology. In order to stimulate collaborations between different centres, the HGF established so-called incubator platforms. Two of those platforms, relevant for this...
The Scientific Data and Computing Center is migrating part of its 200 PB tape-resident data archive to new high-density robotic libraries. The talk will focus on the product/vendor evaluation process and the complex of decision criteria observed. We will also discuss the potential implications of moving from high-performance to high-density robotic systems for hosting active data archives.
The presentation shows the Storage test bed set up that I implemented in 2018/2019 when working at SLAC
to meet the 2020 LCLS2 data reduction pipeline requirements reaching 100GB/s.
This set up is primarily oriented on high performance rather than high availability and is based on Lustre/ZFS with NVMe SSD storage and EDR Infiniband connection between Object Storage Servers and clients...
The interest in the Internet of Things (IoT) is growing exponentially so multiple technologies and solutions have emerged to connect mostly everything. A ‘thing’ can be a car, a thermometer or a robot that, when equipped with a transceiver, will exchange information over the internet with a defined service. Therefore, IoT comprises a wide variety of user cases with very different...
The Elastic stack monitors several systems at BNL. At the SDCC two of these are BNLbox and Globus Connect Server. BNLBox- an implementation of nextcloud- is a service for file storing and sharing while Globus is a platform for file transfers. In this talk I will present how we configured Elastic and its components and how they give us insights into client app usage via log ingestion...
CERNphone is the new softphone-based solution that will gradually be deployed across CERN. Based on open-source components and developments, CERNphone provides mobile and desktop clients and back-end services as a replacement for legacy hard phones and commercial PBX systems. In this contribution, we will describe the architecture and main components of CERNphone, discuss the main challenges...
In April 2020 CERN has formed a working group on business continuity. In this presentation, we will describe the mandate and the direction of the group, and will discuss some of the tools used. Areas for potential collaboration with other labs will be discussed as well.
For several years CERN offers a centralized service for Elasticsearch. This dynamic infrastructure consists of currently about 30 independent Elasticsearch clusters, covering more than 180 different use cases. Using internal monitoring data, a real time anomaly detection system has been implemented,and is now used in production. This presentation describes how the system works, the experiences...
CERN's private OpenStack cloud offers more than 300,000 cores to over 3,500 users with services for compute, multiple storage types, baremetal, container clusters, and more.
The cloud supports CERN's web and administration services and is the tier 0 site in the WLCG.
This update will cover the evolution of the cloud over the past year, and the plans for the upcoming year.
dCache is an open-source distributed storage system for scientific use cases, actively used by large-scale experiments, including within the WLCG community. In 2018, developers started to introduce the concept of storage events within dCache. We are going to present this concept and see how it can be used to trigger automated workflows, with the example of a proof-of-concept implemented at...
The Magnum component of OpenStack is used to provision container orchestration clusters in the CERN cloud, with Kubernetes being by far the most popular cluster type.
This presentation will look at the new features in Magnum and Kubernetes which make it possible to create highly available Kubernetes clusters that are suitable for hosting critical services.
This includes node groups for...
Next generation nuclear and high energy physics experiments are moving filtering and processing tasks, previously done with online resources, to the data center. High resolution imaging systems at light sources and electron microscopes require significant amounts of “online” computing resources to rapidly reconstruct images to allow researchers to make “on the fly” adjustments to running...
Podman is the default container execution platform shipped with RHEL/CentOS 8, and is now also available in RHEL/CentOS/SL 7. It provides a user command line interface that is effectively identical to Docker's, and supports rootless container execution. In this talk we'll give an overview of Podman, our experiences with this software, and a comparison with Singularity and Docker.
