- Packet marking is part of the "SciTags" activity in the Research Network Technical WG. 95 members from 50 orgs.
- Primary goal: allow tracking and correlation of transfers with R&E network provider's network flow monitoring.
- Can associated flows with a specific experiment or activity.
- Provides sites with more visibility into network flows.
- How tags works:
- Packet marking: markers are added to each packet.
- Flow marking: UDP firefly sent out periodically, associated with the flow.
- Network-level collectors at the respective entities collect fireflies or examine packets; hook into the respective analytic platforms.
- See: https://www.scitags.org/api.json
- WLCG:
- Experiment-level systems have knowledge, pass it to FTS3.
- Status:
- Flow marking works with IPv6 and IPv4, hence is the focus.
- XRootD 5.4.0 supports UDP fireflies.
- N.b. - XRootD docs: https://xrootd.slac.stanford.edu/doc/dev54/xrd_config.htm#_Toc88514010
- External-to-storage script based on netstat information.
- ESNet and Jisc/Janet have deployed collectors.
- Simplified deployment was done during the last data challenge. Able to correlate UDP fireflies with netflow data.
- Various additional links to the project on the slides.
- Discussion:
- Does the prototype work with HTTP? A: Yes (Brian thinks it might not work with HTTP push)
- Technical discussion - note that the fireflies are only sent at beginning and end?
- Timeline? DC2023.
- Who is onboard?
- ATLAS is enthusiastic, others are backing / provide interest.
- What about dCache?
- Currently used the netstat version.
- dCache team is aware of the activity but not clear if there's a timeline yet. ACTION ITEM: Need to get input from Tigran.
- How are flows marked precisely? E.g., HTTP headers? TBD.