Choose timezone
Your profile timezone:
TB: EPA/DPP/DAVIT
→
Europe/Zurich
866/2-D05 (CERN)
BI-TB
Present: A. Boccardi, A. Butterworth, A. T. Quintao, B. M. S. Ferrando, C. Roderick, C. Zamantzas, D. Alves, D. E. O. Tzamarias, E. Calvo, J. C. Garnier, J. Joul, J. M. Samblas, M. Gonzalez-Berges, M. Sobieszek, S. B. Pedersen, S. Jackson, S. Jensen T. Lefevre, V. Baggiolini, V. Kain
Main comments and findings
- EPA
- Should do more to involve and consult with equipment groups early on
- Should do more reporting to CTTB
- Is not a CCS project, but an ATS project
- Deliverables will be maintained by CSS/MPE
- CTTB
- Process should be reviewed
- Should do more to capture common needs between groups - such as big data
- GPUs
- Any interest must be signaled to Verena as early as possible
- Very expensive and has 6-12 months lead time
- Offline analysis
- BI relies on delivery of new tool (ref. CTTB), as OAF has been frozen for years as requested.
- Big data / HDF5
- The interest/usage should be investigated (by CTTB, ASF, ?) and a centralized solution put in place accordingly
- CSS will contact interested groups Q2/Q3 2025
- Needed by BI, possibly RF and is also of interest to NXCALS
- The DAVIT application is promising and could be a candidate for sharing
- Machine learning
- There is interest to put ML onto FPGAs
- Sharing of solutions
- Is a complex issue, in particular in terms of maintenance
- More should be done in the sector to enable sharing
- Python
- Is the language to use for UCAP/DPP, GUIs, scripts, ...
- A "How to get started" wiki page is being created
- New Sequencer
- Q2 2025: Jean-Christophe will contact Stephane about requirements
- Will support both Java and Python
- FESA classes will not need to be changed
- Will target HWC first, then IST's
- WRAP
- Will support custom code in the next release a few weeks from now
- UCAP
- Q2 2025: Marcin will contact BI to organize proof-of-concept testing
===============================================================================
Discussions per presentation below:
Verena presented the EPA project
Thibaut: It is important that EPA reports regularly and involves BI early on
The optimization framework and OASIS samplers were not good fits for BI
Verena: I agree, but sometimes it is a balance between involving many parties while also moving forward.
The optimization framework was initially mostly an RF development which evolved into a product of general interest
Chris: Marcin will ensure that WP9/DPP activities are synchronized
Stephane: Accelerator testing is a very good idea, but will start already next year - when will we be involved ?
Jean-Christophe: We will contact you Q2 - we should be able to run everything you currently have in the sequencer
We want to provide 2 additional frameworks (Java, Python) where you can put your stuff
You will not have to change your FESA servers
Thibaut: This can touch on many of our systems, so do you know what you will do first ?
Jean-Christophe: We consider two periods: IST and HWC, we target HWC first
Manuel: Will the new sequencer replace the current sequencer ?
Verena: Yes, but it can do the same and more - also in python. It will be backwards compatible.
Thibaut: We must be involved in EPA as there is a risk that it runs in parallel to CTTB
We do not necessarily want to be in more meetings, but need to be in those which involve us
Verena: There is not enough coordination
Thibaut: There is some frustration on our side as we see things are moving and we are not involved
Chris: EPA is not a CSS project but an ATS project
Manuel: Who will then maintain the deliverables ?
Chris: CSS and MPE will do
Thibaut: We would like more clarification
Verena: There is an issue here, you should have known about the EPA structure/plans a year ago
Thibaut: It is not clear how EPA relates to CTTB and we may have to review the CTTB process and improve the functionality
Verena presented EPA WP8 on behalf of F. M. Velotti
Stephane : What is the difference to AWC - are the dashboards WRAP ?
Verena: It could be, but people do not want to waste time on dashboards
Stephane: So Marcin will make generic dashboards ?
Verena: He will enable that it can be easily done, so people can contribute components/actors
If people are interested, they should be able to join
Thibaut: For BLMs we have done this for a long time, but we want to be better
For tools, Dion can work on it, but it is not his main activity
We want to avoid developing tools which then need to be abandoned
Chris: If you see opportunities in the discussions then signal it
Stephen: But we are not part of discussions, only BLM have been
Chris: You should feed back your interests to WP leaders and they will figure out how to go forward
Thibaut: We should try to redo the BLM approach in our other major systems
Verena: It is good that you are interested
Christos: We should find a way to share effort even if we go first
Manuel: Can we extend the model of sharing contributions to other projects ?
Verena: We need to get much more used to sharing
Stephen: CSS should become a hub for sharing
Eva:
Comment on Zoom:
Anomalies detection is something very interesting for the diamond BLMs, to discover UFOs or other kind of anomalous events.
But in order to get the data available for analysis, we need the detection mechanisms embedded in the FPGA for the data to be captured.
I assume there is some people working already in ML embedded on FPGAs.
I remember having seen something for the experiments.
Is this also under the umbrella of this EPA project?
Verena: You should have talked to me a year ago
We could have gotten money from this, but we did not have a use case. But we will keep it in our list - it has to be treated.
Eva: I did - even two years ago
Comment on Zoom:
Maybe it there is something still that can be done nevertheless...
We have plenty of diamond BLM data recorded, good and bad injections (probably non-labeled but this can change), also some UFO-like waveforms.
Can there be some resource into finding through ML off-analysis the characteristics of good and bad injections, or UFOs?
In a second stage findings could be retrofit into some FPGA logic.
Dion presented BI work for WP8
Manuel: Are you filtering to avoid saturation ?
Dion: We will create a Jira actor, your point is very valid - we will append on existing issues instead of creating new ones
Manuel: How about grouping and summarizing ?
Dion: That would be next steps
Christos: Every link has an id and we create an issue for it - every fault (50-100/yr) will go there
Thibaut: Some issues need quick actions, some are for TS's - issues should be grouped accordingly
Dion: We put a severity value on every event
Stephane: If UCAP actions can be sequenced, can UCAP be the new sequencer ?
Chris: We are trying to clarify this
UCAP actors are currently in Java, but everyone is asking for Python - Marcin is looking into it
It is not easy to synchronize all the actor activities, but we also need to centralize/share
Verena: This brings us back to a sector wide collaboration - but providers have to stay responsible for what they provide
Manuel: But CSS has to be in charge of general libraries, necessary infrastructure, etc...
Stephen: If we make something really good of a more global interest we have to discuss who is responsible
Verena: Yes, then it could come back to the controls groups but it is not sustainable in the long term
Chris: Contributions are not done in a vacuum, there are dependencies, so if another part changes it may require one to follow
Different developers will lead to different user experiences - collaboration is a complicated topic and we cannot do it reactively
We need a plan for controls, so people can see what they could contribute to
Stephen: When we ask FESA team about best practices, we do not get a clear answer
Thibaut: Dion, did you spot missing functionality in WRAP ?
Dion: Yes, but the latest release covers it - still, we cannot provide code to WRAP
Chris: This should become available in a few weeks - there will be a user meeting ~12/3 - state your needs and they may take it into consideration
Marcin: We do not support Python actors in UCAP as blocking calls may lead to data loss
Dion: If there is an official email UCAP actor we will use it
Marcin presented EPA WP9 and DPP
Thibaut: Your request for testing should come soon
Marcin: We aim to contact you in Q3
Thibaut: When do you see the transition from UCAP to DPP ?
Marcin: Not this year as we probably need to extend CMW, RDA, ...
Manuel: It is in theory good to support any language, but not in practice as it opens the door to introduce languages not supported at CERN.
Stephen: You (CSS) should give recommendations on which language to use
Chris, Vito: Use Python
Stephane: A same language also facilitates sharing
Vito: Equipment groups also need to enforce a same language
Daniele: Please try to improve the UCAP documentation
Marcin: Yes, absolutely - we are working on it
Vito: We are putting in place a "getting started on Python" page
Manuel presented BI requirements for on-/off-line analysis and the growing use of ML
Verena : Anyone needing GPUs must signal so asap and it must be thoroughly discussed as these cards are very expensive
Chris: There is 6-12mth lead time and even rumors that Trump may block GPU export
Stephen: BGI may need GPUs, unfortunately very late in the project
Thibaut: What is being used currently for machine learning
Verena: We use GPUs managed ourselves, but we do share - it is a big mess, but we agree it must be solved
Chris: This is a question for CERN and has not been addressed yet
Thibaut: IT also offers the OpenStack - Kacper used it for a month last year
The BI use case for big data has been discussed for a long time, easy for us to move to a centralized solution if this comes soon
This is where coordination is important
Vito: This is a typical service we should provide and falls much in line with what is done for NXCALS
We need to discuss, thanks for reporting it
Chris: Most labs do not store time-series data - they do it shot-by-shot
It would be interesting to see what BI does and compare it to Tango which also has HDF5 storage/viewers
This also can inform how CERN can align with other labs
Manuel: We are very willing to collaborate, but it has to be driven by CSS
Vito: Thank you for being a good ambassador for BI
Javier presented the DAVIT application
Jean-Christophe: Do you support PostMortem data ?
Javier: Yes
Manuel: There is nothing BI specific in DAVIT
Vito: This is very impressive - is everyone using Pandas in BI ? Can you invoke a Pandas query and then show the result ?
Javier: We are not yet that advanced
Manuel: You can run a couple of actions at the end
Vito: We are exploring Pandas on Pyspark and how this can be combined with calculations
Diogo: Dictionaries are also very useful
Thibaut: Do you have FFT in DAVIT ?
Manuel: Yes, it is an option for the plots
Chris: Very nice work - are other groups using DAVIT and/or producing HDF5 ?
Manuel: No-one else uses DAVIT, but perhaps RF use HDF5
Chris: How can we capture the big-data needs ? It is a pity if no-one uses DAVIT.
Thibaut: This could be an improvement for CTTB to capture this - we need more transparency on what groups are doing.
Jean-Christophe: We can organize an ASF (Application Software Forum) to ask about HDF5 interest
Thibaut: I think you need to contact every group
Stephane: Also SoC will start producing big data
Verena: This also touches on simulation data sets
Vito: We have a collaboration with IT for HI-LUMI in NXCALS and we will come to see BI (Q2-Q3) about anticipated needs for the HI-LUMI era
There are minutes attached to this event.
Show them.
