>> Recording in progress. 
>> OK. We still have quite a few people to connect but, hopefully, while I start introducing the session people connect and we don't overrun the session, basically. Welcome, everyone to already the last day of BOOST. I know everyone is probably sad but we have a lot more exciting talks today. I am going to chair the jet tagging session but then we also have the BSM session afterwards and then the panel discussion. First up is jack who is going to talk about "Understanding top tagging with N-subjettiness and prong finding". Please try to stay within the five minutes so that we have enough time for discussion afterwards. Please, go ahead. 
>> Jack: Thank you very much and for the opportunity to speak and present this work. I am talking about trying to understand boosted top tagging using methods with prong finding and suggestiveness. These are tagged using firstly grooming, then is prong finding algorithm that aims to enough the BQQ system within the jets so those three prongs and putting a cut on some sort of jet shape to provide additional discrimination between the single and three prong strip and these methods are used in experiments. I just picked out a couple papers that have used methods in this way. We want to understand the physics of the tagging procedure. We pick a specific example using this wine splitter algorithm to do the prong finding and putting a cut on the end and ratio out the jets and the jets will be pretty groomed with either an NVT or soft drop. We implement this additional step of requiring jets to be within this mass window. First we studied the performance and impact and effects and underlying events using a Monte Carlo study. This is just the signal significance against the cut on tau 32. We found all three of these steps, grooming, prong finding, and subjectiveness cuts provided some gain to the performance. We also found in order to minimize the effect of hadronisation and underlying events that both the grooming and prong finding steps were necessarily. One possible reason why the prong finding step can reduce the effects of hadronisation because it is only [indiscernible] safe but using this splitter method makes it into a fully IRC safe procedure. We did analytic calculations to look at the fraction of the signal and background accounting for the effects of the finite tau effects rather than approximating the cut is small. I know you can see the background and signal, the Red Cross show the results agree quite well with Pythia and the background herwig. From these calculations we know the background and signal are proportional to our pseudo-code factor and for the background, this just has exponential of this logarithm with argument proportional to tau. This is just the approximation here. There is quite a bit more to it in reality. As we reduce tau, this provides suppression. The signal is slightly more complicated. The argument to this log is the minimum of two scales. One proportional to tau and coming from the cut on tau32 and the other one coming from the upper bound that was imposed on the jet mass. Whichever of these scales is smaller is providing the suppression. If we reduce the band on the jet mass nothing happens until the upper bound goes below a threshold proportional or dependent on tau I should say. You can see in these plots on the top row for signal jets, this is signal tagging as a function of M max and reducing that and nothing happens until we reach a threshold which depends on tau and this is reproduced similarly in Pythia. In the case of background jets, there isn't really any threshold to speak of. If we reduce, at max we reduce the tagged background fraction. We can benefit from this difference between the two. We could choose to put our cut on the max around 185 GeV which wouldn't have a huge affect on the signal but it would have quite an effect on the background and remove a significant portion of it. This just illustrates the gain and performance that can be had from lowering this M max. You can see from the red curve we get quite a significant gain in performance with the same signal efficiency. Finally to summarize, we have shown the welcome -- combination of mMDT is a good splitter. Sorry. Both the grooming and prong finding steps are necessary to reduce those non per turb -- 
>> Thank you very much, Jack, for this nice presentation. We have time for questions or comments. Please raise your hand. I think we had someone writing and we have two questions. Maybe Simon, should we get started with your question? 
>> Actually, Jack answered the question. 
>> OK. Then maybe we can ask the first question and I don't know if you saw it but the question was how do the various procedures scale the top mass distribution in particular mMDT grooming on the top part seems very strong. Is the mass peak still around 175 GeV? Or has it moved to a significantly lower mass or smeared out beyond the ungroomed width? 
>> I actually saw this question a little bit before the talk so I had a chance to get a few plots together. On the top here, we have got the ungroomed jet mass distribution from Pythia at parton level with the switch off. On the right, the same thing for grooming with mMDT. And the mass peaks at least within the binning I have haven't shifted. The only difference is there is a small number of events now that are at lower mass. They bin just underneath 175 GeV. I would say no, the mass peak isn't significantly shifted. And then underneath here I have got the same plots but now at hadron level with RSI and MPI switched on to illustrate the difference grooming makes to the mass resolution. Hopefully that answers the question. 
>> Let's wait a few seconds to see if anyone has a follow-up comment. 
>> I guess the other thing is since we are applying the YM splitter it is important the Z condition on the three prongs identified within the jet, then you are likely to fail the tagging procedure in the case that you don't apply grooming and you have a very soft prong coming from the top decay. Are there any other questions or comments? 
>> I see a hand. Please go ahead. 
>> Hi, thanks for the nice talk. I am not very familiar with top tagging. I wanted to ask like could you give a comment on right now in top tagging, everyone, if they are using machine learning, and probably those networks should also use some kind of prong measurement in their network. Do you know what kind of prong measurement they use there? 
>> I don't know a lot about machine learning methods, actually. Maybe someone from experiment or who does design sort of machine learning procedures might be able to weigh in a bit better on that. 
>> I could comment quickly from ATLAS. I assume it will also be in the talks later. We have a talk from ATLAS and CMS both. We also use N-subjetiness which is the one just presented. For top tagging, there is the KT splitting scale that works well. This isn't a prong finder, of course, but I guess those are the most common variables. I think CMS uses M3 but a CMS person should comment. 
>> Here. Jack, maybe I could ask you a very basic question. On slide 4, for example, you focused on rather tight cuts on tau32. I was wondering if there was a specific reason for that and why you didn't extend it to higher values? Is it because of the calculations? Are there problems in those regimes? 
>> You mean limiting the tau to less than .5? 
>> Yeah. 
>> The calculation, I am following a method which I will go through to the long talk slides. So this is following some methods that were first sort of outlined in this paper for tau-21. Same thing here where the calculation gets significantly more difficult for going to values of tau greater than a half and because you require a number of extra emissions in the jet to generate those values without changing the sort of leading emissions. For the significance plots we generated from Monte Carlo, in all cases we seem to get better signal significance for small values of tau so we feel the need to get the values of tau greater than a half. 
>> OK. Thanks a lot. Last chance for any other questions or comments. I am sure there are more. Yes, Gregory, please go ahead. 
>> Is it the transition point at one half and two thirds because you have one extra prong? 
>> I think it is still -- the transition still occurs at a half. 
>> OK. I am willing to trust you on this. I haven't tried it. Thanks, Jack. 
>> Thank for the very nice presentation. Join the Gather Town after if you have questions and you can ask your questions there to Jack as well. Then we can move on to the experiments. First up, we have a talk on jet tagging techniques in ATLAS by Yicong. 
>> Can you hear me? 
>> I can hear and see your slides. 
>> 
>> It is a pleasure to present the latest results from boosted object tagging in ATLAS. Let's start from the motivation of these taggers. High energy particle collisions can result in high pT mass particles. These particles decay hadronically. They are decayed products and can be reconstructed as jets especially when pT becomes higher. But the [indiscernible] can become smaller so the boosted objects are large-radius jets. The identification of this from the decay of high pT particles is very important to physics analysis. Several analyses have used the larger jet to extend their physics output. In Higgs 2BB the decay product of the Higgs boson is reconstructed as VH channel. The decay products of the top quark is reconstructed as a large-radius jets. All the results presented are new for 2021. Here is an overview of the SFWHULG new taggers we have. It has a special resolution compared to before. W/Z and top taggers are optimized using UFO CSSK. Another tagger identifies pross like Higgs to others. -- process. The process to derive these involve similar steps. The decay products are reconstructed as larger UFO jets. Then there is the truth labeling. This is to know the origin of the reconstructed jets. It is done by matching reconstructed jets to truth jets and then label the reconstructed jets with the truth level input. After that, several kinds of taggers are tested to find the ones with the best performance. W/Z tagger is to identify boosted W/Z bosons using larger R jets. We have cut based and machine learning based tagering. The performance is compared. They -- the performance of W/Z tagger with UFO jets and LCTopo jets are compared here. The W/Z tag performance with UFO improved by a factor of 2-4 compared to LCTopo jets. The top taggers are two deep neural networks trained to identify jets from the decay of the top quarks. One is for inclusive top-tagger meaning contacts some of the decay products of the top. The other is for the contained top-tagger meaning it contains all of the decay parts. As you can see from this plot here, the top-tagger with UFO improved significantly compared to LCTopo jets. It is robust against the difference related to physics modeling. Top taggers perform well with a difference of 10% as you can see from this plot. The Digluon tagger identifies processes like Higgs to boson. It is coupled to the Higgs boson. When the mass of A smaller than half of the Higgs boson mass the products can be reconstructed as a small r jet. Then an event level tagger combines the global input with the jet level tagger to return a single score per event. For signal efisficiency of 50% is achievable. Come to the summary. In short, we have a better jet definition with UFO jets and top taggers are re-optimized using UFO jets. We have better performance compared to LCTopo jets. We also have the top taggers. That's all. Thank you for your attention. 
>> Thank you very much for this nice and concise presentation. Yeah, let's see if anyone has any questions. I know there is one on the Google Doc we will go to. Let's see first what the question from the audience may be. I will read the question out loud. Are all taggers moving to machine learning now? Do they demonstrate there are physics missing to understand the tagger? Can theorists help develop new taggers? 
>> Thank you for the question. About all the taggers moving to machine learning which is not all of them because we still have the [indiscernible] and Digluon tagger. These taggers are useful for some of the physics analysis because it only use this few wire. It is simple compared to the machine learning tagger. Because we use fewer wires. We need to configure less systematics. It is easier to understand the systematic effect compared to the machine learning tagger. For the question about the [indiscernible] tagger it is still useful. And then about the few wire taggers demonstrated there is physics that we are missing. I have discussed this question with one of the others and we think the few taggers doesn't miss major physics because in the few of them that we have the larger jet mass and a larger jet wire track which is the number of detector tracks before grooming. With this wire bonds, we already boosted the larger jet mass and the substructure. There is no major physics missing but compared to the machine learning it used less information. About the third question, yes, theorist can help the new taggers. Many of the ones we have used in the taggers no matter what they are developed by theorists so yes, they can develop new taggers. The new taggers might use different techniques but we still need to understand the structure and the between signal and background jet in the substructure. That's all. 
>> Great. Thanks. I could talk a lot about this but I think your answer was very nice. Let's maybe go to the next question because I see lot of hands raised. Andreas, please go ahead. 
>> I have a question on slide 5. The signal efficiency for the top tagging. You see a huge difference between say [indiscernible] and in particular on how this behaves as a function of pT. 20% differences at low pT. How do we deal with this when you calculate scale factors and systematic uncertainties later? This is different in the simulation. Like is it counted as uncertainty or do you have ways around this? 
>> I am not an expert on this question. If one of the other authors is here, I think it is better for the author to answer this question. If not, then what I can read from the published note is the major difference here is within 10% and this is not a very large difference. 
>> Yeah but I guess you want to be better. That's my question. One question is this is more machine learning tagger. Is this behavior now more complicated than before using machine learning tagger? Or is that similar? 
>> I saw Josu raised his hand and he would probably like to comment. 
>> Hi, yes, about the scale factors. So previously for the top-tagger we were including the modeling differences as they related to there scale factors. But for this now we see this not so tiny difference and we are thinking about deriving dedicated scale factors depending on the [indiscernible], for example. They derive scale factors to correct the pT Parton showers and different scale factors to correct the [indiscernible] scale factors to know the sensitivity to the difference. The difference in efficiency that you see here is not going to be translated into [indiscernible] on the scale factors. 
>> OK. And is this effect bigger than previously? 
>> Yeah. It is slightly bigger than before, yes. 
>> Do you know why? 
>> We are basically making the study right now trying to see if maybe the [indiscernible] because now the training used for mc tupal is harder and removing more. Maybe this is related to having more sensitivity to the Parton showers. We are basically making the study to try to understand this difference. 
>> OK. Thank you. 
>> Thanks, Josu. Then I see Clemens raised his hand. 
>> Thanks for the nice talk. I have a question on the same slide and then even the same plot but it is not directly related. When you compare here Pythia 8 and harwig 7. I think I remember this substructure in ttbar measurement that you performed I think like two years ago or so. I was wondering whether for the tuning of the underlying event model if you use this measurement or if you think there is potential the modeling or the difference between Pythia and herwig can be improved if you use that. 
>> I think this question also might be one of the authors can answer. 
>> Maybe I can quickly say. We haven't used that measurement in our Monte Carlo yet. It is certainly we would like to take these into account into the future. I think there is potential. We are trying to disentangle different effects and what is coming from the new jet collection grooming algorithm and how much does this really depend on this. There is a bit more to understand. But certainly, yeah. There needs to go more of these measurements into the tuning of our Monte Carlo. It hasn't been done yet. It is a long process to get this in, unfortunately. Is 
>> OK. I understand. Thank you. 
>> Great. OK. I see one more hand from Christina. Maybe if it is a rather quick question we can go to it? 
>> Sure. Just a very quick question. I guess in this Digluon tagger slide, how do you validate this? I guess the study in the performance and simulation is clear but are there plans or has this been published in terms of validation of this tagger in data? I don't know if this is the right talk to ask about this. 
>> Um, from what I know, not yet. 
>> Maybe I can... maybe I can quick Leo -- quickly elaborate. We can calculate uncertainties with calculate the individual check on the uncertainties. That's one of the advantages of using track and objects you have full uncertainties on like that. That's the sort of one aspect of the uncertainties and validation and in that sense, of course, for the background, the modeling and things like this that's being done in data. There is an interesting aspect of the training that uses this adversarial process to minimize the differences between the data and the simulation in the training process to make sure it learns features of the background that are in the ground and not just in the simulation. It is quite sophisticated in that respect. It is not an answer to your question because I think there is no standard candle that's very easy that, you know, that we would like to fully validate the signal but many aspects of it there are plans for that. I would encourage you to read the note. It has more details on this. 
>> OK. Thanks. 
>> Great. Thanks, everyone for the discussion and thank you, Yicong, for this nice presentation. If people have more questions, try in Gather Town afterwards and we can discuss more there. Then I would propose that we move to the next talk and now we are going to hear about boosted jet tagging in CMS and our speaker is who'll -- Huilin. 
>> Hi, everyone. I don't need to convince you boosted jet tagging is important and a very powerful tool for experiment list. Basically for boosted jet tagging I know there are too many handles we can rely on. One is the different radiation patterns and the other is the flavor components. Thanks to new machine learning approaches we can now better explore them simultaneously and significantly improve the performance. In CMS we have explored and studied a variety of boosted jet tagging algorithms. These are mostly summarized in the plot. We have algorithms ranging from more traditional cut-base approaches using jet substructure and I mean we also have like algorithms now using machine learning, you know, either using like low level inputs or using high level inputs. And more recently we have developed like beyond the classification of the jet mass regression used at the end there. For the baseline algorithms in CMS we are basically using the jet mass of the grooming. For grooming we typically jet mass and for the pronging we use N-subjettiness. For the machine learning I think what are most commonly used in CMS with the DEEPAK8 and the ParticleNet algorithms. They both use the low level features. Going down to the particle flow candidates. The second vertices of the jets. They are multi-class taggers meaning they can do top tagging or W/Z Higgs tagging in one go. We also actually have subcategories targeting various decay models. Since you can aggregate the scores, you can transform them, these are very vert versatile taggers. For ParticleNet, the jet is represented as a set of particles in place and we have work for tagging. If you compare the performance, you see DEEPAK8 significantly boosted performance and the tag one efficiency is more reduced by an order of magnitude compared to traditional approaches. And the ParticleNet even improves significantly compared to DEEPAK8. One of the quite important features, if you want to use the taggers in practice, is the correlation with the jet mass. This refers to the fact the tag one jets start to modify the jet mass shape and they essentially if you have tag reflection, you can have artificial [indiscernible]. This isn't a problem but if we can reduce it it will be more desirable for analysts. One approach is DDT and performing a transformation on the tagger score. Or you essentially apply a cut that is dependent on the jet pT and mass and therefore you go make a selection with a constant background efficiency of X % across the pT and mass range. Another approach is ad ver -- adversarial training. The mass decorrelation is based on a dedicated signal samples which the residence in the sample is no longer fixed. It is a continuous spectrum and a wide range and therefore in the mass spectrum it is similar between signal and background and then the training doesn't develop any of this. Looking at the performance basically among these taggers, the MD shows the best performance and gives the smoothest spectrum in compared to the other approaches. And the notes we put together are recently on mass regression. The goal is to use the neural network to reproduce this and avoid any sculpting of jets. This is used in the particle detector. We also use a low level input as candidates and vertices. Also this training. For the regression target, we use the generated particle mass or the pole mass of the signal. For the background, since the pole mass like quarks or gluons are zero, so we use the soft top mass to avoid this. If you look at the performance, you see the regression brings improvements. The resolution is much better. It also gets rid of the tails especially those at small values near 0. The improvement is also consistent in different jet flavors. Here for the Higgs and the light quarks and more important we see that this response and resolution is very stable versus the particle mass and also we don't see any scalping in the QCD sample. I just want to touch on the calibration. It is very important to calibrate these taggers well before you use them in your analysis. For top W tagging we use the standard muon sample. For Higgs B B or CC we use gluon splitting. For the mistag it is done with dedicated control regions and this analysis-specific. Just to give you a feeling I show scale factors for top tagging and W tagging. You see things are well modelled in simulation. That's my talk. I think at the time I just want to say that the efforts we invested in developing this paid off. It will lead to impressive improvements in several analyses which you will hear more from. Thank you. 
>> Thank you for this nice presentation. I would propose we get started maybe with questions from the audience. We have a few in the Google Doc for sure. Josu, please go ahead. 
>> I have a quick question on slide 5. Why the signal efficiency at 65%? 
>> Ah, OK. This is because so we apply a mass cut. It is written here. This is applied only on the numerator of the efficiency definition. That's why you have like inefficiency not reaching one. 
>> OK. I see. Thank you. 
>> You are welcome. 
>> Gregory, please go ahead. 
>> Thanks for this talk. Since you are on that paper as well, I am wondering whether you compared there ParticleNet with the [indiscernible] net Fredrick talked about on Monday? 
>> We haven't studied this with CMS but we are interested in exploring especially if we look at how the robustness can be improved. 
>> OK. Thanks. 
>> Thank you. 
>> OK. I was about to propose to switch to the Google dock -- doc but I see Max raised his hand. Let's go with him. 
>> Thanks a lot. For the talk. It is very interesting. Could we go to the mass regression, please? 
>> Yup. 
>> Maybe the next slide I think is the one I wanted to ask about. The interesting thing here to me is, I mean, you have have the soft drop mass that has clearly one prong because you cut away another prong and that's why its mass is 0 with respect to the true mass. The correction factor that you are applying is you are essential taking 0 and inflating to a large number. 
>> It is not a correction factor. We are predicting from network and directly predicting the mass from the network. It is like a direct mass value prediction. 
>> OK. So you are replacing -- it doesn't -- you are not actually using the mass as an input? 
>> No. No. Just using the particle level. 
>> It is like it is directly learning there mass. 
>> Yes, exactly. 
>> OK. But I mean if you threw away half the jet, by using a soft drop, I guess the question is are you running this on the pre-soft dropm inputs? 
>> No, we are running on the ungroomed jets. 
>> OK. So the mass here the improvement is coming from the fact you haven't thrown away half the jets. This is sort of obvious and makes sense why it improves that way. OK. Great. Thank you for the clarification. The other question I had was the target. So you, for the target, you used the pull mass instead of the truth jet mass. I was curious about the motivation there. I understand it is always nicer to use the pull mass and for something like W/Z and top those are well defined to some extent, you know, maybe in top it is a little bit less well defined but, you know, in jet calibration, we also calibrate to truth jet or something like that in order to avoid questions of like, you know, is it fully, yeah, if you are missing energy, for example, or missing particles outside the cone and all sorts of things like this, there is complications and you are sort of fixing to the truth mass and for the pulled mass you assuming you are not missing particles outside. I am curious if comparisons were done on this and why that target was chosen compared to other targets like the truth jet mass. 
>> We studied this choice of the target for the signal. Since this algorithm was more or less developed in the contest of analyst we explored instead of the pole mass we used the same definition as the become ground and what we found was that in terms of the sensitivity to the analysis, it is actually better if we use the pole mass because it gives better resolution for the signals. I mean, like the concerns they have about this modeling effects, we actually studied in the resolution in data using like semi leptonic ttbar using the w peak and what we found was with that uncertainty things agree pretty well. I mean the scale factors on the resolution is just a few percent. We think this approach is reasonably well modelled in the simulation as well. 
>> OK. Very interesting. Thank you. 
>> Yeah, I think there was one other question related to the regression on there Google Doc that asked about the low mass peak you have when one of the prongs is being removed and if you tested different soft drop parameters basically rather than going with the regression and I guess the same question is for ATLAS basically if there is anything that the theorist can help with maybe to improve this rather than going directly to the machine learning techniques. I don't know if you want to say anything about that. 
>> Yeah, so, I think two questions that start from the first one about the different configurations of soft drop or other approaches. I think we didn't study this much in CMS but of course this is something very interesting to explore. For the other question about like how theorists can help since now we have machine learning and tagging, I think my perspective is there are actually still a lot that theorists can help with. One would be the most important or very important and that would be really to improve the modeling of like W shower et cetera, and event generators because now with machine learning we are actually getting into like more lower-level stuff. We are actually becoming more sensitive to the modeling. If you can improve the modeling in the generators, I mean, like on one hand we can improve the performance and on the other hand, have more control of the tagger and radio systematics. I think something that would have a very strong synergy between the theoretical and experimental community is something I would like to see very much. 
>> Great. Yeah. I agree a lot with that statement. Thank you. 
>> Thank you. 
>> OK. Are there any last minute questions maybe quick ones? I don't see anything. Thank you very much again for this very nice talk. This was a very interesting session. I think we are reconvening in 12 minutes with the BSM sessions. Please join Gather Town in case you have more questions. Thanks, everyone. Santeri diboson LHC 
>> Hello, everyone. I lope -- hope you can hear me. I am be chairing the last session of talks. This is on the BSM physic searches. We have three talks and we will begin in about 30 seconds. Let's see if Santeri is connected already. Hi, Santeri. Would you like to share your slides already? 
>> Hi, can you see and hear me fine? 
>> Yup. 
>> Perfect. Let me share the slides. 
>> As a reminder, five minutes for the lightning talk and we will go through the questions from the audiences or the talk for 10-12 minutes. Yeah. OK. I think you can start now. 
>> All right. Very good. Let's get started. Hi, everyone I am Santeri Laurila from CERN. For the next 5 minutes I will try to present you five brand new CMS results on new physic searches in boosted diboson final states. In five minutes it is impossible to be comprehensive so I welcome you to look at the longer version and the analysis documentation if you are interested. Here is the list of the five new results we will cover. All of these are searches for new physics at high diboson invariant mass and they are used essentially AKA jets with ML jet classifiers and boosted jet techniques as you would expect in this conference. These are all based on the full Run-2 dataset. The first result I would like to present is a search for axion-like particles in the final state where we have haddronic decay of Higgs boson. Here we set results on the rez net production -- resonant production. For the first time we set limits on the non-resonant production in the final states and the limit plots you can see here. Then the second search is a search for a heavy resonent X decaying to what's lighter than the standard Higgs boson. Usually these kind of new physics we will mess with the standard diHiggs searches because usually we have cuts on the standard mass. This is the first search of the LHC for this type of process. Also here we set limits as you can see as a function of both the heavy resonant mass and the mass of the light scaler boson. Then moving on to the third search. This is now a similar search but this time assuming the standard model Higgs-boson mass. This search was also performed with 2016 data previously but now using this new generation DEEPAK8 tagger that was just discussed. We are getting a major sensitivity gain compared to the previous result. And also there is some new so-called semi result category where one of the two Higgs is boosted. Here you can see the upper limits for spin 0 and spin 2 resonants and I overlaid to 2016 limits roughly so you can see it is almost an order of magnitude gain in the sensitivity so much more than you would expect just based on the luminosity increase. There is also a similar search but this time just one of the two Higgs is decaying to bb bar and the other decaying to w for tau leptons so we have either one or two leptons in the final state. This is also something that was a search that was performed previously but now a new addition in this new version of the analysis is this dilepton channel with two leptons and large missing pT. Also here we set limits for spin-0 and spin 2 rado -- radions. The final search is our brand new results on boosted non-resonant bb net diHiggs production. This bb production is an interesting process because it is the only process that we have to probe these quarks decoupling between two vector and two Higgs bosons. We know if the value of the coupling deviates from the standard model the majority of the signal can become boosted as shown on the plots and there cross section can increase so much these could be observable already with the Run-2 dataset. This is exactly the process we target here. This is also the third CMS analysis to apply the ParticleNet machine learning graph neural network architecture that was just described to identify the Higgs boson candidates. It is used for the jet reclassification and the mass regression. Here are the results. For the first time we are able to exclude the value of this quark decoupling. This scenario where the quark decouple takes the value of 0. The scenario of vanishing coupling which fixing the other couplings to the standard model values. This is also in sensitivity a major improvement to the previous analyses. These were the five analyses and now let's discuss. 
>> Thank you very much, Santeri. So let's see if there is any raised hands or questions from the audience. We can take a look at the question in the Google Doc. I don't know if you have seen them, Santeri. 
>> Certainly. Whatever you would like to discuss we can do that. I will switch to the longer version of the slides. 
>> First this question I am going to read out. With extensive use of machine learning for diboson searches how amendable are they on RECASTing or setting bounds? Do you plan to release helpidate with the performance of each machine learning based tagger for each resonant? HepData. 
>> Right. My answer to this is maybe something we can discuss or go deeper in this also afterwards. My first answer to this would be that this RECASTing of the search results is always a bit of tricky business. Partially so, of course, with the ML taggers but also otherwise. With other selections, without plugging in the signal sample coming from specific models, it is hard to say if some selection efficiencies or trigger efficiencies even change with respect to the sample that was used in the analysis. I think the best approach here is if you have your favorite model and you think our search would be sensitive to probe that then, please, get in contact and then let's try to simulate that process and add it properly in the next round of that search would be my answer. 
>> OK. Anyone would like to respond to that? OK. The next question. This is respect to gain of double B. What about with respect to single B because double B is inefficient at high masses. 
>> Right. Indeed here the number that we quote is with respect to this double B tagger which was used in the previous analysis. It is true that if you have a very high pT jet and basically the two -- the 2B quarks or B hadrons are overlapping more and more the higher you go in pT. At some point, it just requiring a single B tag subject might give you better efficiency. I think these type of ML taggers that actually look at the whole jet and the correlations inside it and the different magnitudes of overlap between the 2Bs. I think by design they should outperform the algorithm whether it requires exactly one or two subjects which is B tag. 
>> It makes sense. Out of point of comparison, if you want to make a more convincing case, there is 0 background or very low background at very high masses so single B tags are often sufficient. This is how the ATLAS XHH very boosted analysis does it at the highest possible masses. A single B tag is enough to kill the background. And [indiscernible] is the one who developed that analysis back in the day. I think it would be another interesting point of comparison in terms of signal efficiency. B-tagging will always kill you with two B tags. 
>> I agree. That's a good point. In the paper, we can see if we can quote also some number with respect to a single B tag. 
>> There is one more question on slide 7 and 8. It is a bit long. Your average mass plot shows a deficit of data for three bins around 40-60 GeV immediately followed by access of two bins around 60-80 GeV. It seemed like it may be the background may be estimated with the wrong slope in that region. Since you use an [indiscernible] function to determine your transfer factor when estimating the QCD background, were others considered? And how was parameterization taken into effect? 
>> That's a long question and requires a long answer. If I tried to make it short the first thing is that so this deficit here around 50 GeV is it a fluctuation or a problem in the background model? These plots here are just basically a sum of the different categories and a projection along one particular axis. There is one other plot in the paper which I put here in the backup which unrolls these 2D distribution we use. On the X axis you have the average mass but we show the histogram for each MHH bin. And if you look at this, you see -- so this under fluctuation is present-only in a few of these categories. Then if we look at this total number of more than a 100 bins I think all the fluctuations seem to be compatible with what you expect. Basically only around 5% of the bins shows two sigma up and down fluctuations as you would expect. There doesn't seem to be a major modeling problem. Another thing I would like to add is in a search like this you don't want your background estimate to be too flexible and you don't want to change it after unblinding. If this would be this effect that we see, if it would be an overfluctuation instead of underfluctuation here than it would be a potential signal and we would not want our background model to accommodate. 
>> I hope this answers the question. 
>> OK. Thanks for that. I think we can do one more question, a quick one. You mention a 2D fit to dijet mass -- and then go on to show multiple exclusion limit plots with a fixed and variable dijet mass. Was the fitting background in slices of these variables? 
>> Yes, indeed. The fit is per formed as a function of the average and dijet mass. Then we set the limits as a function of two variables -- the resonant mass H and A which is strongly correlated with the average and di-jet mass. 
>> OK. Thank you. There is one more question but I think we don't have time for it. Maybe you can respond to it in the Google Doc. 
>> Sure. 
>> Thank you again, Santeri, for the nice talk. Moving on to the next speaker. Josu? 
>> I can see your slides. 
>> Hi, everybody. Here I want to present you some slides that benefit from boosted techniques in the final state. Here you can see this is the talk of the analysis and things I want to cover and links to the papers in case you want to see more. First, I will basically briefly describe the large R jet collections that are used in this study. We have this standard which is called LCTopo large R-jet which are clusters calibrated to the energy scale so we apply the procedure to remove the pile up and use the combine mass which is basically mass which is calibrated with calculated combining and another using tagging information and by doing this we can have a very good mass resolution in the entire pT range up to 2.5. We have the TCC large R jets which are jets built from track-calorimeter clusters. You have the figure here that shows this. The tracking information is used. We have in general very good reconstruction performance at high pT. The tuning procedure is applied for this ttc large R jets. We have a couple collections allowing to perform the bottom up approach. Max commented about this in the previous talk. We can prepare from the very well known constituents to the high level. One is called track assisted reclustered jets. We use these two construct the mass and variables. These are scaled at reducing the information and we have large reclustered jets that exist on reclustering and on the smaller jets input in an a lot of jets. Moving on to the searches that I want to talk about. First, I want to say that in general, unfortunately, the standard model expectation described the data. These searches are putting exclusion limits. The 1st one is plotting the dark matter model including the scaler mediator and the bosons here. You see if the mass smaller it will be boosted and we can have four prong jets in the final state. We can benefit from reducing and applying cuts to this subjects to remove two and three prongs. You can see this is a function of the Higgs scaler. You see in this region with the mass of the scaler it is small compared to the right hand region. The limits have improved these four tag # -- tag candidate. These are predicted by models trying to show the weaker scale problems. We have the two particles produced here and we have a stage with many different particles and just the Higgs boson tops and the bottom. At least for the search benefit from producing multi-class boosted targets which as a multi-class tagger allowing to identify B jets, Higgs jets and top jets. You can see the score variable here in the middle plot. Yeah. By using this multi-class we can identify events which have two Higgs, one Higgs or one top in the final state and final region and compare with the model. You can see the top and particles are excluded. Maybe 1.60. OK. Then we have a heavy resonant. In particular, these coupled to the quarks in order to have a resonant or two boosted. It is much larger than the mass of the top. These benefit from the target which is basically showing different structure variables and we correct this by means of the scale factors. You can see the special limits on the right for this. We have searches with particular boosted Higgs bosons in the final state. Usually the Higgs is assumed and this is useful because it has high branching ratio and also signal ratio thanks to the B-tagging techniques. For this case, I mean for these searches are using the B-tag targets associated to the larger Higgs candidate to identify the Higgs. This is very useful because I mean this variable track jets which has the radius increases with the pT allows to reKRUSHTH -- reconstruct more than one subject. For the dark matter searches in particular this allows to improve or at least have to sensitivity to masses of this A scaler 1TeV and when the difference of the masses of the two particles is quite high so this means the Higgs is going to be very boosted so we benefit from having these track jets. Also in the single vector like B quarks where we have heavy B-like particles and the mass is higher so the Higgs is going to be boosted. In particular this is excluding couplings above .3. We have another search for the scaler leptoquarks. This is constructed using including kinematic variables and structure. But also with that we can derive an even level variable and define different signal regions depending on the value of the BDT and the data. Particularly for this model we can exclude the mm leptoquarks with less than [indiscernible]. Finally, I want to show you this search for high mass resonant with W Y and plus photon. These are predicted by many theories. You can see here for example this search benefit from using these TCC jets to identify the two-prong expected from W and C jets. You can see the comparison between the solution of these two variables of the LCTopo and TCC. On the right you see the exclusion limits derived in particular for the single model and heavy resonant production. Finally, by improving the performance, it has allowed increasing sensitivity to the cross sections and more improvement is coming in the future so, please, stay tuned. Thank you. 
>> Thank you, Josu, for the nice talk. Questions from the audience? If not we will go to the Google Doc. I see there are two questions. I don't know if you have read them but I will read them any way for you. For the DNN that was constructed for the ttbar resonant search, these are correlated meaning the DNN has information of Monte Carlo for tagging that is difficult to understand physically if at all. Are there efforts to use less correlated observables? EFP and excluded subjects, et cetera, that would still ensure a broad coverage on the phase space? 
>> To answer quickly the second question is yes. We are studying different top cutters using les correlated inputs. And then commenting about the question. I think that the strong argument is too strong because there were dedicated studies to select these structure variables. By defining different groups and comparing the performance of the tagger by including one by one each of the variables.
We observed that the performance was significantly improved if one of these variables were added to the content without the variable. This means the DNN is in fact learning from adding new variables. It is true there are correlations but we are including new information by computing these variables and at the end we will end up having this set of variables which is a minimum set of variables that we have found to perform better. Yeah. And then we are also studying them into the performance between the data Monte Carlo using these ttbar events in order to take into account the possible differences in the Monte Carlo and data too. I think this is a carefully studied in this case. 
>> Thanks for the answer. Any questions from the audience on that? If not we will move on to the last question. Santeri, would you like to ask the question yourself? 
>> Sure. I can go ahead and ask. So, yeah, basically, with run-3 coming up soon, probably CMS and ATLAS are thinking about their trigger menus at the moment. I just wanted to ask first of all are some of these analyses limited by the trigger efficiency at the moment? And secondly, do you have dedicated boosted jet triggers that you use for some of these analyses? 
>> Thanks for the question. Yeah, so for these analyses, or at least the topologies where we expect to have boosted objects in the final state, we are basically using this singularity triggers which consist on trigger at high pT trigger level. This is basically fully efficient for resonants with masses above 1 GeV which is where the boosted levels play around with leptonic channels, for example, which is applied in the low resonant masses. For non-resonant productions where you expect to have in general particles with pT comparable with standard model processes, in this case, yes, there are dedicated [indiscernible] for the strong work in this direction trying to include better triggers.
But in general, for this, yeah, for this analysis which is looking for boosted objects, yes, using these large R jets will be fully efficient for pTs over 500 GeV are OK. 
>> Clemens, would you like to ask a question? 
>> I have a question on I think slide 5 of your slightening slides. I was looking at the limit plot. -- slides. One of the versions is that it flattens out pretty quickly. It is overall rather flat. I was wondering since you go down to an mt of 800 GeV, is it actually still advantageous to use R equals 1 jet? Or shouldn't you even be looking into larger radius jets? How efficient are you actually still in tagging, you know, the top of the Z or the Higgs at the low pT that would result from a T that is the amount of 800 GeV? 
>> So, yeah. I mean in general, especially for the tops here in ATLAS, we are considering boosted tops for pT above 350 GeV. In this constraint, there are certainly quite a large amount of tops which are not reconstructed within the lower jets. In the pT region we will benefit by increasing the [indiscernible] produced. But then on the side, include more problems. For example, the streaming procedure has to take into account how this can affect these jets. I think this collection which is basically candidates I think with the sequence of 1.5, I guess, was also used in ATLAS in round 1. At the end, we observe the performance were similar. Even having lower radius. This one particle to one but maybe including all variables for target and we can achieve this better performance and also at the end this radius is better to be used at high pT because, you know, the effective side of the jet is going to be used by pT. Yeah. At the end, basically we are using the standard R jet that's equal to 1. 
>> And the effect of flattening out towards high masses is that because you don't have any -- basically, you are not selecting signal events any more? What is the reason this is happening or do you just have one large bin at the end? Why do the limits flatten out so much from let's say 1200 to 1600? You don't see improvements in the limit. In principle, background should go down towards higher masses and signal efficiency stays the same and the limits improve. 
>> I think for this particular set because you are requiring so many objects in the final state it is statistically limited. 
>> OK. We will move to the next talk. Xudong, we are ready for you. 
>> Can you hear me? 
>> Yes and I can see your slides. 
>> I will give a talk about heavy BSM standard model research. Several searches I covered in a long talk and in the short talk I will focus on tri-boson searches. This search is motivated by the extended warped ED model. And the traditional boson process is on the [indiscernible] -- this has a larger contribution. The -- the case of the two Ws. In this analysis, we choose the long channel with the branch of 42%. According to the ratio reader mass and the W mass, we have two cases. Merged case and resolved case is formed. Someday -- the jet mass, based on the AKA jet multiplicity, the jet mass and tagger scores, define six origins. For the jet mass below a 100 GeV, double taggers, and if the jets mass is above 100 GeV, we consider a jet as defined as the W Higgs tagger to tag it. The red box is the definition of the tagger and you can see the combination of the W and the Higgs and 4 scores. The samples are broke down into four based on the JMS and the jet pT. These four plots show the distribution in the samples. If you look at the ratio of the data to the Monte Carlo before correction there is clear mismodeling in the first and last bins. This measures data of the bins. So, we define the calibration measurement referred to measured method. This method is compacted. If you are interested, you can go to the details. You can see the details in the backup of the long slides. The next step is to calibrate the signal jets. The correction for the merged radion jet is a bit complicated because they have different [indiscernible] for the merger jets. We can have four quarks inside. Or three quarks inside and a lepton inside. This top right plot shows the distribution for different conveners. We don't have standard model kindles. We need special treatment. What we did here is to require one extra quark inside the fully top to form a four-point structure. You can extend the model to mimic this. We can apply scale factors here and we apply the scale factors to W2. The final differences are considered as systematic uncertainties. Here are the results. You can see the plots in the signal region and signal region 4 is the most sensitive region and has no excess over the background prediction you observed. On the right, limits see the WKK and reader mass plane are set below 3.4 TeV. I quoted for the merged case and allowed for the log case is 130I 3.6. To sum up the machine learning taggers benefits the boosted jet physics a lot and now they have a more powerful tagger in ParticleNet. There is no evidence for a few fix and there is no upper limit on the mode parameters. Recently, the path for the troubles in full hadronic state is public so stay tuned. That's all. 
>> Thank you for the interesting talk. Questions from the audience if there is any. OK. If not, let's take a look at the Google Doc. I think there is one question for you. I am going to read them out here if you haven't seen them, Xudong. Some of these searches, referring to your long talk, some of these searches seem to be approaching an absolute upper bound in searches for resonant and LHC. How much further can these searches be pushed even with run-3 or LHC data? What is the next frontier for BSM searches using boosted final states? Are there rare but low mass processes that can be probed? 
>> OK. For the first question, I would say it provides opportunity to implement novel triggers and newer approaches like more machine learning base taggers. The new search can [indiscernible]. Although I don't have the values. And for the frontier, perhaps the next up, I would say to probe a more channel intensely like the case to BB it can also include Higgs to WW or Higgs and the W can decay hadronically or help tonically. And use ParticleNet and it had significant improvement at low and high mass. These channels I would say the signature of Higgs to WW is similar to our tri-boson search. There can be some opportunities there. 
>> Thank you very much. I think that's it from the Google Doc. Yeah, Max, please go ahead. 
>> Thanks a lot. I was focusing on this slide again because it is up but it is also quite interesting. With this four prong top, you using as the proxy for your four pronged R for the calibration. Could you describe how, I mean, I guess it is top quark with an additional quark or gluon inside. What studies did you do on the labeling of that? How many events do you have Rick -- like snit -- like that? I am curious on details there. 
>> OK. For the extra quark, of course it is not from the T and B. It requires a pT bigger than 250 GeV? And the percentage of this component to the top object depends on the pT of the top. 
>> How do you discriminate between the four and three prong top? Just applying your four prong tagger to a sample of top ttbar or something like that? 
>> They don't have overlap but we found [indiscernible] are similar. We consider this. We only have one scale factor for the t3 and t4. 
>> I see. Thank you. Matt, would you like to ask a question? 
>> Yeah. No, I just have a question about this tagger as well. More of a curiosity. I am wondering whether the tagger performance behaves on the flavor of the extra prong whether it is a quark or a gluon? 
>> The effect I am trying to remember. 
>> Yeah, I think -- we didn't observe any significant differences. But it so big it is hidden by statistical effects. 
>> Thank you very much. Maybe one final question before we end this session? OK. If not, thank you very much, Xudong. And to everyone asking the questions. That ends our session. The last one for the talks. We have about 14 minutes until 5:00 for the panel discussion. See you all in about 13 minutes. Thank you, everyone. 
>> Hi, everybody. My clock just ticked over the hour. 
>> I wonder if the organizers want to say anything or should I introduce the session? 
>> Please, just go ahead. 
>> This is a panel discussion which we are very lucky to have a panel of five of our participants representing various sections of the BOOST community. We are hoping to have fun kicking the ideas around in a relaxed environment. There are some questions to get us started. I will as moderator select a couple to get us going but I think the discussion can be pretty much carried on from there. I would like to ask the panelists to say a couple words. Maybe Cari first? Cari, I can't hear you. I think you are still muted. 
>> Sorry. My Zoom appears to be struggling. People are moving in slow motion but hopefully you can see and hear me. 
>> I can hear and see you and your guitars. 
>> Hi, I am Cari. I am a PhD student graduating this yeart a Harvard. I work in high energy theory but a lot of my work has been focused on how we can use tools like collideers to understand and collect data in different ways to search for generic beyond the standard model new physics. 
MODERATOR: Thank you. Felix? 
>> Hi, I am Felix. I am a postdoc at Berkeley. I work on the theory side. We are doing a lot of QCD calculations relevant to jet physics both at the LHC and MPP collisions and also here in the U.S. the LHC collide will come up in the future. I have always been doing more recent work on machine learning as well. 
>> Thanks very much. Leticia? 
>> I am moving to CMS in a month but I am interested in peak QCD but mostly at its high temperature limit and applying jet substructure in order to characterize this space. 
MODERATOR: Thanks very much. Loukas? 
>> I am Loukas. I am at CERN. I am working on jet physics and machine learning and also in [indiscernible]. 
MODERATOR: OK and finally Max. 
>> Hi. I am Max. I am a saf scientist at triumph in Canada. I work on ATLAS behind me. I have worked in the BOOST community on a number of topics from quark gluon measurements to standard model measurement and these days I do jet reconstruction with machine learning and search measurements involving di-Higgs and searches for SUSY also. 
>> Thanks. I should say my name is Jon Butterworth. I worked on BOOST since the start of the LHC. Less so recently but I really enjoyed listening to what everyone else is doing. I am looking forward this discussion. We have heard a lot of really cool new results over the last four days. The first suggestion on the Google Doc is, you know, not resting and given what we have heard over the last four days, what are your hopes and what should we try to accomplish before we meet in Hamburg? What would your favorite results be in a year's time and what you think the community should be focused on. I guess I will ask all the panelists to comment. If you don't have anything to comment on a question that's fine. Just passover. Maybe we can start in the same order. I will rotate the order as we go through different questions but let's maybe start with Cari this time. 
>> There were a lot of questions. Is there a particular question I should start with? 
MODERATOR: The first question I was saying can you maybe imagine yourself in a years time and what should we be focused on and trying to deliver? 
>> Cari: I feel like -- can I go last? 
MODERATOR: Of course, you can. Loukas, are you happy to go next? 
>> Loukas: That's fine by me. Something that I think would be great is to be able to have more better improved generators, particle CERN models, and it is a bit challenging. It is challenging to, for example, to develop the taggers and algorithm. It is a challenge to calibrate them and to, you know, to remove the uncertainties. Particularly now that we don't expect Henley -- huge improvements from the development but on the other hand, we want to gain an order of magnitude. It becomes, I think, much more relevant to try to have a closer collaboration between theory and experiments. One of the areas that I think it has been -- these uncertainties. I think this is one of the priorities. 
MODERATOR: Do you know how close we are to like higher order precision on parton showers, for instance? Do you think that's something we might get this time next year? 
>> This is probably for theory. 
MODERATOR: Definitely. Leticia now. The order will be random because the Zoom screen is moving people around. Do you are anything to comment?  -- you have. 
>> This is my personal wish but I think many share it. We are finally able to link one of the modifications of the substructure in the jets to something very fundamental via comparisons to the theory. I think this is enriched finally because the theory has been progressing. There are certain regimes that are calculateable and if we are smart we can link those. Finding the hardest splittings within the jet for instance and trying to probe large angle scatterings which will be proof we can scatter off free quarks and gluons on the strongly coupled. I think this would be really, really key. 
>> Cool. 
>> Yeah, no, I think the way jet substructures have shown a light on the high density QCD is a surprise to a lot of us and clearly a fast growing area. Does anyone want to follow-up on that or to add their own thing in? 
>> Max: Maybe a question. Why hasn't it been done? Maybe, you know, that's a statement but, what's the challenge? What's the roadblock preventing that? 
>> Leticia: ' 'It is easy to ask why this hasn't been done but for the heavy ion community it took awhile to understand the need to measure observables that are calcuably. I think it straggled with many years with observables that looked at correlations between tracks. Finally, we have the tools. This emerged like three years ago or so. We have to understand how to control the background because, you know, in heavy ions we have a huge background for the jet pT but also for the prongs that we select with the substructure. This is one of the main difficulties we have. I think we have been working on that for the next years and this could be more or less under control. And then I think now it is a time to focus more. Also, the theory on their side. There are people like Gregory that are more qualified than me to speak about the progress of the theory but I think we are close. 
>> Felix: I think as Leticia mentioned, the observable experimental considerations but I think it is just a very difficult question from the theory side where I would say generally there is a lot of disagreement about how to interpret results in heavy ion collisions. They are more like theory-oriented results that try to use observables that are well understood in the vacuum and try to go from there and see what they can understand about the medium. There are people in the community who say you can't do first and only approximate modeling. It isn't that I agree with that but there are many that think it is the case. It just a too difficult system to control or extract information from it. Now that we understand a lot of the substructures observables much better in experiment and from theory in pp  coalitions -- collisions I think you can try to make progress. It is definitely a very complicated environment where you have like the large background, flucuating background, and it is unclear how that affects variables and how to set-up a consistent framework to study those. I think it is an open question. 
>> Jon: Maybe I can abuse my position as moderator to ask a question related to this. I think one of the things we learned, I think we are still learning, buts we learned a lot of on the LHC, on ATLAS CMS, is the power of set rate measurement from interpretation as cleany as possible so you have -- there is a lot of activity particularly in BSM on reinterpretation of results. I have a feeling, because of my involvement in things like that, that this is a journey that heavy ion experiment is going down now. The axis on the plot are making more physical sense independent of the model to me. Is that a fair reflection? Or am I mischaracterizing what's going on there? Does that make sense? 
>> I think the observables are better defined and I think in this sense what you are saying is the truth. 
>> I would agree with that. I guess how to interpret and connect it to, you know, some fundamental -- I don't know. Some quantity set of factorization and theorem is the question. Some people say you can't really do that with all the medium response that's around. I think anyone has to try to make a bit more quantitative progress to see if these effects matter or, you know, if it were really true you can only do some modeling and then you have to like interface parton shower with hydro and you have to do that. That interface can be problematic and there are different interfaces to do this. Things can change a lot. We have data that makes sense and hopefully you can interpret it in a useful way. There is a lot of disagreement between the different approaches. 
MODERATOR: Cari, want to comment? 
>> Cari: It might be a pivot from that answer but I definitely agree with Loukas' comment in the beginning and I want to go to the end because I think it might kickoff a different discussion that might be contentious but that never happens appear -- at BOOST. As a theorist what I would love to see in the community, and we saw a bunch of these fun talks in this past session of how we can look for beyond the standard model stuff by looking at jet substructure and trying to unpack unboosted objects and things like that. I think in theory we could make a lot more progress of trying to figure out what is reasonable in terms of the kinematic space that we have and just the topological features if theorists can see more of what's being collected and what's going on. I am a big proponent of open data and not just necessarily the data but high-quality Monte Carlo simulations. Something we as theorist don't know how to do particularly well if you are just an outsider like me. In terms of where I think the community can make a lot of progress and what can be helpful to theoists trying to come up with innovative ways to suggest experimental searches I think open data would be an excellent thing I would like to see more of in the future or in a year from now. 
>> We are going on a journey starting with Loukas' comment about generators through the heavy ion. The models are very much involved in the physics we get from it. Back to actually opening the data and the models up and seeing and doing. Anyone have further comments? We are moving toward one of the other questions which is as a theorist what are you missing from the experience and that's more open data. Anyone want to add more to that? 
>> Max: I would extend the invitation to join the ATLAS collaboration. NAIBTS -- maybe that's a bit trite. Loukas you were probably going to say something more serious. 
>> Of course, yes. But my question was actually is -- if the theories have access to open data and simulated data, if I understand correctly, it will help improve the communication and tuning of algorithms and the implementation of the developments on the theory side. If I understand, this is one of the limitations. For example, ATLAS had a very nice measurement using the plain. Maybe I missed it but I am not sure how much [indiscernible] was done from the theory side using these results? I may be completely wrong. How useful was this? Or how can we improve from your side. Open data can be one. 
>> Max... I think there is ongoing work in ATLAS to extending this open data idea at least for specific projects. I can't say too much but there is ongoing work which should be interesting for this and especially in the context of tagging and things like this. I think, yeah, I mean they are working with individuals like you said the SDA mechanisms to make sure that what we provide is useful to the broader community at the end of the day. But, yeah, I guess that question goes back to you Cari, and as Loukas said, is it the actual data or the simulation and being able to mess with it for any purpose? The whole dataset is a bit difficult to reveal from a size perspective even. Yeah. 
>> Yeah. So to respond to Max, some of my collaborators and mine a couple years ago from MIT and Harvard we did that open data project where we did an actual search and then, of course, our systematics and all of our error bars were huge because we are not nearly as good at this as experimentalists are. When we did that, I feel like we may have rippled some feathers but to address Max's concern of what do you want and what are you going to do. I feel like the conversation between theory and experiment could improve with this idea that sometimes I think theorists are too pie in the sky. Obviously if you remove the background and you can make a super detector and have no uncurrents and new physics will fall out -- uncertainties. That's not helpful to anyone. But if theorists don't know the shortcomings or the triggers even people are running at the LHC which isn't perhaps too uncommon for the average theorist to know these things it is hard to do realistic proposals. Since it is not so obvious what kind of models we ask -- we should be hunting down or focus our attention. For example, our strategy and similar to a lot of strategies that were presented, was we look for something with a high pT because that's far from the background and easy to find. Our paper, the point was to stress test that idea. If we had found new physics that would have been a mess. We would have done something wrong if we found something and it had never been seen before. The point of the paper was not to show look at this. We did this analysis. Take us seriously. The point was to say experimentalists, we did this and you can do this so much better. The purpose of open data is for theorists to test their own ideas so we aren't feeding experimentalists pie in the sky expectations. I feel like the open data is more of a reality check for theory rather than like theorists skipping all the hard work experimentalists need to do to have a publishable result with no error bars or something silly like that. 
MODERATOR: Do you think the STA system is addressing that need? Or do you have reservations about it? Sorry, for those that don't know, this is a short term attachment to experiment where you therefore get access to some of this. 
>> Cari: I think it addresses some of the ideas. You know, the beauty of the open data release from CMS that we were using was it was kind of older data. It was data collected in 2011 and we were looking in 2018-2019. You know, the STA is a really good system but it is also a bit of a bottleneck. Maybe it should because with great power comes great responsibility was our punchline for open data so maybe there should be bottlenecks so people aren't doing crazy things constantly. But I think it would be nice too if like you have an idea, and you are like maybe this kind of search strategy could work, you could think about it for a couple weeks and if it doesn't pan out you didn't waste a bunch of people's time on the experiment. If you have a specific idea I think an STA is an excellent idea. If you want to explore it as a higher barrier. 
>> I have reservations about theorist losing net true neutrality. Anybody else have a comment? 
>> Max: Yeah, I think it is a great point, Cari. Especially understanding, you know, what's the limiting factor. You mentioned triggers, for example, and things like that and knowing that. OK. We can always clarify the triggers a little better in the papers if that's an ambiguity. I guess you are pointing out they are often experimental aspects of the reconstruction that delphies or something doesn't get right. So testing your idea to see whether this particular new thing is going to work really requires the high fidelity simulation and things like this. I think we are always getting better. In a year's time we will see what we have gotten. You are in STA and Matt tells me. I remember you were the proposal for that. I hope in a year's time we have more. I don't know if it will ever be quite everything but there will be, hopefully, progress. 
>> I think expecting everything would be unreasonable. 
>> We have to leave something for the rafter, right? Good. Thanks. 
>> Can I maybe ask a question about this STA program that I don't know maybe other people know. How does it work? Once you have access to the data can you publish anything you want or does it have to go through approval process if you work on it as a theorist? 
>> Max: In ATLAS you join a project and propose the project to individuals at either your institute or anywhere you have friends. Then you have access to data in the process format usually not the raw format but you do have access to data usually. It is explicitly discussed usually. And then your name would be on the author list of that paper. It is not something that you go out and do your own analysis completely. You would be working with the collaboration on a well defined specific journey. 
>> Yeah. 
>> There is an interesting discussion going on in the chat which I would like to pick up on in a minute and comes back to Loukas' accommodate about comment about -- comment about improving the Monte Carlo. I have been recently involved in LHCC kind of discussion forum about open data but about what needs to be provided from a measurement so it can be maximally useful to theorist. I have a feeling that what you are after, Cari, is not that because that's more using a measurement to interpret in the context of the model. What you are interested in is developing new measurements which I think is special to the boost community and I think therefore the needs will not be met by the kind of discussion I was talking about. Did that make sense? Do you want to comment on that? 
>> Yeah, I think that that's definitely a long -- along how I would interpret that as well. Certainly both are useful and I don't want to discredit the theorist that would be useful to. 
>> For the kind of work Leticia is talking about then the maximum information from the measurement is probably what is needed. This isn't an instead. It is an as well and not an instead. I will go to anyone on this topic of open data and accessibility of the open data and if not, I suggest we move on to -- we are about halfway through the session. Please, people, if you have, I think, I can see hands when they are raised and I can certainly see the chat in front of me. I would like to come back to this business of improving the modeling and the Monte Carlo which was raised initially by Loukas' in the first answer. Matt, want to give a quick summary of what you are proposing on the chat? 
>> Hi, yes. I can do that. In the chat, I guess we were talking about and this really is stemming from the comment from Loukas but I think it also connects back to what we discussed after Jennifer's talk on the first day of BOOST where there have been a lot of substructure measurements made over the course of Run-2. There is a whole Twiki page of them. It is very long at this point. There have basically only been like two very exploring studies done using Monte Carlo to do any tuning which is a primary motivation on many of the measurements we are publishing and one of the applications we hoped it would be useful for. I think there is some discussion about people seem interested in trying to do some tuning which I think awesome and a bit of discussion about where the studies should take place that cross experimental boundaries. The measurements are public but there tuning studies -- the tuning studies whether they are taking place within the collaboration or a neutral zone. I suggested the LHC electroweak Working Group. There is a jets boson subgroup where a lot of substructure and tuning discussion happens. There might be a better place, I guess, so there were studies done at [indiscernible] two years ago but that doesn't happen very often. I don't know if there are better settings for that to happen. Or if we should just have a boost tune group that does some studies. In principle all the infrastructure and measurements are public. There is no reason we can't just do it as a community. 
MODERATOR: Anyone want to comment on that from the panel? We have done these and had proceedings from BOOST that have done that job at the level in the past. That's certainly something we should consider. Maybe you guys on there organizing committee should consider that. I am a member of this MCNET, European Union action thing which I am still allowed to be in despite being British. It brings together the office of some of the herwig and Pythia and Madgraph and the full final state ones. It is actually increasing touching on heavy ion stuff as well. There is a lot of interest in this. Maybe there is collaboration and approaching that group as a community is worth it. You can engage in that. I know there is a lot of work going on around EIC preparations. I see Leticia nodding. Want to comment? 
>> No, I was just agreeing. 
>> Agreeing is good. Felix, want to say something? 
>> You mean regarding specifically the IC? 
>> Is there an appetite for kicking off tuning activity targeting the substructure and should it or should it not include heavy ion stuff? Any thoughts on how it might be best facilitated? 
>> I think it would be nice to have that. I think it goes together with some of the things that I think would be very interesting measurements in the future and that's to focus on sort of like soft aspects of jets and like jet substructures that are sensitive. I think a couple observables have pretty good control about non perturbative effects. If the measurements can be combined especially the soft aspects of jets I think that would be useful to do. 
>> Just as a reaction, I think, Matt, I appreciate you bringing this up again. It is the broken record of BOOST I think in some ways. We have said this a lot of times. Then it never happens. I think it is the sort of thing that it is not going to happen until someone makes it happening. Andreas is here and you are mentioning having some kind of challenge/suggestion or, you know, making a Working Group as Simone pointed out sort of from the past and this could be follow-up proceedings within a small author list or something like that. But I think there is enough people here that I think, you know, an ex -- expertise is available. I don't see why not. I think it would be great to have people from ATLAS and CMS on it. People need to stand up and say with their own time that this is valuable enough. And until THAPT ns, it is not going to happen, I guess. -- that happens. 
>> Was the exclusion of ATLAS deliberate there? 
>> No, excuse me. That was my bias. 
[Cutting out]
>> And also mention of the loom planing and at low pT that is very valuable. 
>> Good. Thanks. Well, maybe, Matt has given himself a job here or at least one of the outcomes of the discussion can be to mandate the local organizing community of this virtual BOOST to see if this can be taken forward and come back with ideas. Some kind of opt-in study where everyone who can contribute does. I see Max has his hand up so let's take it as a yes. Not his hand up but his thumb up. Very good. Excellent. I have a question which wasn't on the Google list but I would like to get there panelist's view on. Ge a -- the value of BOOST as a concept in terms of boosted objects decaying to hadrons came because we had a leap in energy. We had a leap above the electroweak scale and had Higgs and W and Zs moving quickly and needed new techniques. We are not seeing another leap in energy any time very soon for sure. I think that, you know, we are now looking at a higher lumi era where things are moving more to precision. I would like to know what you think are the limiting factors on precision and boosted limited? What needs to be done depending on the measurement and theory, 20%-ish precision maybe down to a few percent or better. Anyone got comments? Max use have your hand up as a preemptive strike on first choice. 
>> This was the question I was hoping someone would ask. I think thinking about where we are going in the H-LHC with not more data and energy is important. Two aspects I want to point out that are the right ones to think about. Where in the phase space of EFTs where a lot of searches and measurements are converging on EFT formation because it is a general framework. And so, yeah, I think finding the places where BOOST matters for efts is fun and interesting. I don't think we have this pinned down fully. The two I think should get more cratet is the ATLAS different ttbar -- ATLAS differential ttbar. There you are setting unique limits on coefficients and things like this and starting to participate in the global conversation but the boost has a home there. It has a natural area to home in on. The other one of course was Loukas' analysis or the CMS analysis on the HPPVF. That's not an eft exactly. It is a different framework. But it is a unique measurement with boosting is the only way to to anything -- to do anything on that coupling. That's great. If we find those things we have a lot of things to do. There is certainly going to be portions of the eft phase space that are destined for the BOOST community. Finding them, understanding, you know, where the non-resonant signatures are going to give us boost is going to be an interesting longer term prospect. In terms of precision, I point out again, the nice techniques of the ttbar differential cross section analysis that gave these jet scale factor measurements trying to extract corrections from the data to hone down the uncertainties to really small amounts. I think that's a place where these kinds of techniques we have to go beyond just calibrate and forward scale factor and do things like that to make it even better. I will stop there. 
>> Loukas, do you want to comment on that since your analysis came up there as well? 
>> Loukas: I would like to second Max's comments. We have to understand where we put the focus. We don't have jumps in the energy so I think one way is to improve the precision and uncertainties. That why I mentioned at the beginning, I think this is a top priority and requires collaboration like the collaboration between both theory and experimental communities. Just to comment about the Working Group, or you know, hackathons. I think, you know, I think if something were to happen, you have to have a smaller group of highly motivated people that can devote some time and I think it is high time to do this. I think we would benefit for both communities. 
MODERATOR: We have the kind of experimental techniques. We have the improvement of the modeling and Monte Carlo back to where you came in from the beginning. Is it the per preterbative or the soft QCD side or the parton showering? Do we have a feeling for where the main limit factor is? 
>> At least on the algorithm at the time it was different. The algorithm I am working on and developing it is on the parton shower. There can be significant differences. The improvement is so large we can be generous with the systematics but now we are in the state where it becomes more and more important. 
>> Felix or Cari, one of the theorist want to comment? Or both of you? Go ahead, Felix. 
>> Felix: I agree. I think there is quite a few questions about all the different aspects like the perterbative side and the non-preterbative side. One thing like the soft jets we can use universality aspects that can be derived from first principles. I think that's a nice way to improve and I think it was mentioned in the first talk of the workshop how you can improve the non-perturbative aspects and if that's actually the case. -- non-perturbative. I think there were like some proposals where one can maybe have an even better understanding of these types of things than for the observables that were already measured. In very rare cases, one can even compute the leading hadronizations with still being safe. I think there are some observables but we have better control of those and one should see if they work with measurements and theory calculations. I think that can be used to immolate the calculations and basically check and compare to parton showers. I think that would be a useful thing. I think in terms of precision, I think another interesting thing is to think more about charge or track-based jets. I think it was mentioned a few times also. I joined the workshop where I think it both requires new measurements that can be done much more precisely but also from the theory side I think one has to like think of a lot of observables were not done or worked out how to include moments of the track motions. I know for a lot it isn't done or a bit more complicated. I think that's a way one can increase precision and do sort of more precise comparisons between calculations and parton showers. I think that would be an interesting thing to see in the future. 
>> Thanks. Cari, want to add anything or pivot again? 
>> I mean it is going to be a pivot. I have a question, like a question for experimentalists. When we go into this high lumi update, we might be able to explore more rare event topologies. Would people be amendable to the idea of like trying out kind of crazier triggers that might look at something a little bit more rare and weird since there is just so much more data to look at? Is that something people are considering doing? And like therefore maybe we can use new event shape observables since we have heard a lot about interesting ones at BOOST this week. Would that be something that people would be interested in considering in the high-lumi era? 
>> I see Max and Loukas nodding. Loukas, go ahead. 
>> First of all, you know, definitely we are considering it. I think [indiscernible] is considering improving the triggers. Tagging techniques on the trigger side. Also, more let's say generic approach is like normal detection on the trigger side and trying to find some patterns that are not expected. I think this was discussed in the experimental overview. These are things we are definitely related. Of course, we are putting a lot of the record to include them already in run-3 which will be used a lot to gain experience. This is the best we can do to understand the models.Al 
>> I think that's a big yes. You would say the same for ATLAS, right, Max? 
>> I don't speak for ATLAS but we are going from 1 to 10 kilohertz and it sounds like there is a lot of room. A lot of that room goes to maintain existing thresholds because pile-up makes our lives awful. You definitely squeezed something in there. That's the interesting thing. The question would be, and you know, an event shape observable and there is always questions of can you get out of level 0 or level 1 and things like this. I think that's probably the hardest thing with the hadronic one especially. You know, in ATLAS, we will have a lot more information at level 0 for calriometers and CMS has baby P-flow at 40 megahertz. A lot more information is going to be available and we will have to see how good these trigger systems can be to keep up. But I expect the level 0 and 1 will be the hardest but seems our technology upgrades might get us there. 
MODERATOR: Thanks. OK. Anyone got any further comments on that topic? I guess the issues are somewhat different in the heavy ion community. I will say I do remember the near side ridge from CMS being the example of the trigger that ATLAS forgot to put in. We couldn't confirm it because we didn't have the events. Despite our best efforts there can be exciting things we miss and we are always keen to hear ideas we are missing. 
>> Probes from heavy ions will bring new channels I was going to say. For example, boosted objects in heavy ions and hadronic decays of W or sets and using those as time chronmeters was proposed originally by Gavin and others. This is more for FCC with the top but perhaps before one can start doing something directly with the Ws decaying. I think new channels will be opened with varying probes. 
MODERATOR: I am looking at the questions and looking at the clock. I want to save the -- first of all, I don't see any hands up but if you have questions, please, put the hand up. I want to save the one about where do you think the next ideas in boosted physics will come from until the end. Let's look at the one -- we have had a lot of talks on machine learning and started early on with talks that use machine learning. We had a summary of course. What to you see is the biggest not yet exploited technique for that content? Maybe we can stay with Leticia? 
>> In the context of heavy ions I think ML starting to be useful and will probably be useful in trying to identify conveneing jets and separate them in order to be able to probe low P T -- pT and not to stay with only high pT jets. That would be a domain for sure. And in classifying events from less are kind of things that I think we could start being explored soon. 
>> Do you say that are greater reprecision and reproduceability? Or do you think they are equally on the theory side? 
>> Didn't get that. Sorry. 
>> If you start classifying using machine learning, do you see issues with interpreting that in terms of the theory? Or do you think that's manageable? 
>> I am not sure about that. Felix, what do you think about classifying events according to... 
>> Yeah, I think it depends how one goes about doing that. But I fully agree in heavy ion we haven't started exploring these. I think it is one of the things where we should use those. There are a lot of things we don't understand fully from first principle. I think one should combine with explainability and interpretability and there are complete sets of observables and one can use those to train, for example. And maybe try to identify certainly observables -- certain -- that are promising. I think if one combines it with a degree of explainability I think it will be useful to do that. At least that's sort of one of the thoughts I had. I think it is definitely something that would require new ideas. It is so far not very much explored. I think that's one of the areas where I think machine learning still has a lot of potential to explore a lot of new things and requires new thoughts. 
MODERATOR: Thanks. Just going along my Zoom board, Loukas, do you have any thoughts on this? 
>> No more than what has been said. This is a very important topic. 
>> We have to understand better what is being learned in the field. 
>> Thanks. And Cari? 
>> This is definitely above my pay grade to discuss machine learning. Where I have seen it look really promising to me, and I guess that's particularly relevant to my work, is again seeing it in these reconstructions and tracking and things like that because I am really interested in looking at the shape observables so seeing machine learning, like all the excellent progress that's been made about machine learning to really understand the events at track and particle level. I think that's super cool. Obviously there is tons of other applications. 
>> You say pay grade and I think it is actually above mine as well. I am -- I find I haven't worked on machine learning. I find myself worried about it. Those that know boost know I am always worried about this kind of thing. I worry when we use it to define observables. When -- I think using it to define things at particle level I am totally comfortable with and I will do whatever I can. But when you start defining observables in terms of a machine algorithm I get nervous. Probably unjustifiable. A lot is borne of my own ignorance but it wants me to be able to at least think about it. Max, want to add anything? 
>> Max: Maybe this is not going to make you more or less uncomfortable but I think machine learning reconstruction is something I am thinking about a lot these days. We have seen from ATLAS, for example, how much going from our LC only jets to UFO jets really gave to us in terms of using more information and paying attention to things like multiple particles overlapping and getting the best reconstruction possible and that really opened the doors to a lot better fidelity in jet reconstruction and finding the best ways of using the track and calometer information. I know it has been done in CMS. But the UFO approach had quite a few new things brought to it. You see papers on archive and so on and people think about getting machine learning doing similar things. I am thinking it will be fun because sharpening the inputs and using track inputs like Felix is talking. We can use neutrals too if we are smart. It will take the graph neural networks first impression pile-up we heard about earlier and take the machine learning particle reconstruction in more sophisticated ways and I think that will be interesting and that will give better fidelity for the complex observables. 
>> That makes me feel good. Cari, I think you were going to come back. No? Leticia, do you want to come back on the subsequent comments? 
>> No maybe I can add another interesting problem that might be tackled with machine learning is the flavor dependence of jet quenching because the quenching signal sometimes looks like gluons are more quenched than quarks and this needs to be clarified. You could say I can go and go to a very high pT and tag heavy flavor jets and that's a quarks but if you look to the low pT regime then yup. Maybe ML helps. 
>> Thanks. OK. We have five minutes left. Athere is five of you. Let's have one minute each on what you think the next big ideas in boosted physics will come from. What are the biggest outstanding problems in your opinion? Can be completely bias, arbitrary and just focused on what you care about. Anyone ready? 
>> I can go first if that's OK. Basically, sort of one random aspect, but one thing I think is quite interesting that goes into the direction of the discussions we had about machine learning. There were some proposals, for example, the group that worked on flight last year where we combine generative modeling with a form of inference. You can train models on full event data and try to learn something from the parton shower. There is different ways you can go by little information or putting theory in the neural networks or invertible neural networks and trying to run physics from it. This combination of generative modeling with inference can teach us interesting things about physics. I think that's quite at the beginning. I think one can really explore very interesting things in this direction. I would be one of the things I think would be interesting to learn more about. 
>> Certainly interests me I have to say. Who wants to go next? Can I ask? Leticia, would you like to say? 
>> Yes, the biggest outstanding problems of jets and heavy ions in my opinion, again, the biggest problem that if I had the money I would like to solve is -- cutting out -- this is only coupling liquid that we probed and whether we can say we are scattering a [indiscernible] and really probe the underlying QCD within the liquid. This is a problem. I think this is the one billion problem so to say. And then substructure can help us here. And then it also happens to be a regime like a large transverse momental -- momentum interaction and there are less dirtiness. For me, this is the key point to tackle. 
>> I have to say my interest in heavy ions have been growing exponentially over the last few years. Max, you are next on the Zoom screen if that's all right with you. 
>> No problem. I think the next ideas given the context of the LLHC and ATLAS going into high level data. It is time to think about where does boost meet eft and where do we interact with the rest of the community in terms of making precision measurements at the highest energy scales and finding the operators and the eyeing n -- Eagan vectors that we can help with and drive forward and you know, classify or quantify lack of physics or the presence of the physics. 
>> Cool. Thank you. And Cari? 
>> Yeah, so coming from a BSM stance, looking for new physics, what I think is really interesting is doing sort of these more robust search strategies where we look at event topologies and understand if it is something we know or might be knew. I think there is a lot of fun to have in development of event shape topologies at the event or global level will be fun. If I am doing super selfish for theorists developing the tools it would be fun to test them on out on open simulation. 
MODERATOR: Thank you. And finally, Loukas. 
>> I think it is important like I said at the beginning with the measurements and have to do some reverse engineering. First of all good news in standard but don't have to convince you that there is a black box. And because if theorists do they need to develop verifiable observables and I think they do to understand. In order to have gain with a similar approach it means there is something more there. I believe the new tools can deepen our understanding. I think looking ahead and next year we have snow mass and I think it is important to also, you know, test, explore techniques and developments of the future hopefully at some point. I think these are the areas that would, you know, I think look for improvement. 
>> Thank you, everyone. Just on that one, I think we were extremely lucky ATLAS and CMS built brilliant detectors without knowing they would need do boosted reconstruction in advance. Luckily we can both cope with it. The next brand of colliders will have a heard start because they will know in advance they do need to this from the get go.  Just to remind Matt and others all these depend on improving the simulations in the data we already have. I think we should not forget that is something we trying to initiate. I have to thank all of the Panelists for a lot of interesting contributions. I have really enjoyed this meeting. I will hand it back to the standing organizing committee. Thank you very much, everyone. 
>> Thank you very much. This was really great and interesting and, yeah, thanks to Jon, and to all the Panelists. I will briefly stop the recording. We only have a few more things to say before we close BOOST for this year. Let me quickly bring up the slides. A big thank you for attending boost 2021. On behalf of the local organizing committee you can see at the bottom of the slide. Yeah. I mean it has been an online boost. I think it has been different than last year. We tried to make it different. We have now about 15 hours of recorded discussion and presentations and they have all been captioned. It they are not yet available they will be available soon. You can find them linked from the agenda and the time table but you can also get them from the CERN videos platform using the keyword BOOST 2021. Matt did something really nice. We have all the captions so that means we have all the words that have been spoken and while he is using a word cloud that shows what's been the big topic of boost this year. Maybe with the exclusion of today because that still needs to be added but you can see we talk a lot about jets and I also find it kind of cool we have think different in here. I think really a lot of nice talks that we had and lots of good discussion in the spirit of BOOST. Now, we still have the Gather Town room open if you want to join. There are a couple rounds of scribble going on which is kind of fun. You can chat with people. If you want to join directly after this talk and exchange with others and continue the discussion we just had, again, big thanks to Anna with help creating this space. Final words of thanks new a few individuals -- now -- and bodies. First off to Connie Potter who helped with the particular organization and fighting with the CERN administration in particular. That was really important. And then also to the BOOST international advisory committee for advising and the financial support and CERN for the financial support. And of course, we should not forget about the session chairs or the speakers and the panelists we just had. The captioners. It is crazy what they have been doing. Some people talk really fast, like me, sometimes at least, and then you have all the physic-specific vocabulary and that's really hard to caption but I think they did a really great job and we would like to thank all of you for reviewing the contributions. Some of you did that in advance. And of course, contributing to the live discussion itself. That leavess me with there last slide. See you in Hamburg hopefully next year. If you have feedback on this year's boost, feel free to drop a message to the email on the Indico or on the slides. Any final words? 
>> See you in hamburg. I am looking forward to it. 
>> Indeed. Same here. Robin did bring up in the chat we need motto for BOOST 2021. 
>> Looks like BOOST different? 
>> Let's take it up from the word cloud. There you go. 
>> Jet different maybe? I like that. 
>> We can think about it in the Gather Town. We don't have to die on this hill now. We can do a good job. [Laughter] 
>> Indeed. 
>> Thanks, everyone. That's a close for BOOST 2021. Hopefully see you in person next year in Hamburg. Bye, everyone. 
>> Thank you, bye. 
>> Thank you. 
>> Test test 
>> Hi. OK. I mean I guess we are all here. That's great. OK. Where do we want to start? Obviously the priority is probably the talk, right? Giordon, is there any value in quickly flashing there state of the slides or are they the same state as we left them? Maybe just as a reminder.