Minutes
- TTree to RNTuple reconversion (update the converted files from Jan 2024 with current version on the ROOT EOS space)
- Clustering bug fixed
- New version has the full information from the TTree input, including unused aux trees
- Recreation of multithreading plot and read rates
- Contain latest XRootD fixes
- Scalability difference of RNTuple wrt. TTree may be due to I/O saturation (how to confirm that?)
- Mesaurements: should ensure known state of cached/uncached
- We should measure the actual I/O load on the infrastructure - due to HW/OS-level caching & prefetch this can be much higher than what is requested by the application
- The small scalability issue when converting AGC input with 77 nodes is understood -- runs in the bandwidth limit of this particular EOS installation
- MT versus MP penalty is likely due to the XRootD client (multiple connection pools vs one connection pool)
- PILOT: saturates at 8GB/s (10 processes), with better pattern could be 40GB/s
- For ALICE: 20GB/s vs 350GB/s
- Due to the bad read patten, the read-ahead tuning is actually not super relevant
- RNTuple developments
- Report RNTuple I/O numbers in RDataFrame
- Cluster prefetcher based on memory limits, not on number of cluster limits
- Create an AGC dataset with 10x cluster size / page size: goal is 1MB reads
- Understand idle periods of the network in MT RNTuple remote reading (tails of RDF MT scheduling?)
- Improve RDF RNTuple MT schduling: truely cluster-based, not file based
- HTTP access discouraged because files get opened and closed frequently and monitoring output is huge
- Possible next tests
- With XCache
- With reduced dataset, reading everything from the RNTuples
- Concurrent AGC runs
- Run AGC on Ceph-FS
There are minutes attached to this event.
Show them.
