[Enrico Fermi Institute] 12:21:19
So slides again. I need to share them.

[Enrico Fermi Institute] 12:21:32
Do you want to do these? Or here? I can go through them.

[Enrico Fermi Institute] 12:21:37
So one other thing questions on the charge was what metrics should be used to decide with a work for this executed efficiently, both in how we acquire the resources and and also then how we operate the work through zoom is it different is it efficient to get a certain resource to basically spent the effort to to get

[Enrico Fermi Institute] 12:22:01
it That's a cost to get it, and then actually to run one out, work through science and acquiring in this context means 2 things: one and like one, is is to actually get access to the resources which on Hbc: And Cloud we're talking about the proposals, and hpc competitive proposals, where you put

[Enrico Fermi Institute] 12:22:22
it. Usually this year at the moment, is his yearly proposals.

[Enrico Fermi Institute] 12:22:27
You have to follow a certain procedure of every Hbc.

[Enrico Fermi Institute] 12:22:29
Facility is different. The exceed slash access There's like an umbrella organization where you can ask for time on multiple facilities.

[Enrico Fermi Institute] 12:22:37
These in one proposal, but others are unique for you.

[Enrico Fermi Institute] 12:22:43
One facility, and on cloud is either you just pay a you go, pay whatever this price, or on demand, or spot, or preempt.

[Enrico Fermi Institute] 12:22:53
So whatever the instruct is called, but is publicly available.

[Enrico Fermi Institute] 12:22:55
Pricing to everyone If you show up the credit card you can get it, or like what athletes is doing right now.

[Enrico Fermi Institute] 12:23:04
A subscription based on a negotiation, basically commit, we commit to a certain amount of money, and you get a certain block of resources with limitations and rules, how you can use them and how things are going on and the second part of the acquiring part is the extra provisioning once someone gives you

[Enrico Fermi Institute] 12:23:23
the key. Basically, here are the resources you actually have to figure out.

[Enrico Fermi Institute] 12:23:27
How do you actually tire them into our systems so that you can make use of them?

[Enrico Fermi Institute] 12:23:34
So at the Hpc. Level. It's like things like batch, Hbc: badge, queues unit of provisioning a number of notes scheduled Our policies all of that comes into play because it's all different than what we I used to on our own resources.

[Enrico Fermi Institute] 12:23:51
That we own where we have a fixed quarter. We say you get 4,000. Course.

[Enrico Fermi Institute] 12:23:55
Okay, There might be a 20 four-hour way to test me back to other people.

[Enrico Fermi Institute] 12:24:01
But eventually, if you provide a stable, I'm basically a sufficient amount of work will always give you 4,000 courts.

[Enrico Fermi Institute] 12:24:09
That's different on the Hpc. It's like you don't have any guarantees there.

[Enrico Fermi Institute] 12:24:14
And then cloud cloud is less problematic in terms of provisioning, because you you pay over your money, but you still have depending on what pricing model and what rules you follow.

[Enrico Fermi Institute] 12:24:26
You can still have to deal with contention with insertion.

[Enrico Fermi Institute] 12:24:29
Certain regions side which depend on the size of the regions.

[Enrico Fermi Institute] 12:24:35
Activity of other customers, What certain instant types you request, and so on.

[Enrico Fermi Institute] 12:24:40
Yes, and then, once you have the resources and they are available, and you provision them and they're integrated.

[Enrico Fermi Institute] 12:24:49
Then you look at what metrics are interesting to determine whether you actually operate efficiently.

[Enrico Fermi Institute] 12:24:56
There standard one we use everywhere. Cpu efficiency gpu efficiency.

[Enrico Fermi Institute] 12:25:05
It's Basically, nothing. It's an open question. We We don't have anything that measures how efficiently we use the Gpu

[Enrico Fermi Institute] 12:25:14
On the cloud what it eventually comes down to is the dollar per event, or the dollar in paid per hsu 6.

[Enrico Fermi Institute] 12:25:21
You get. Hbc: there's no direct all right.

[Enrico Fermi Institute] 12:25:27
Cost associated. So the outlayers 0 in terms of monitory, but of course, is not free, and effort.

[Enrico Fermi Institute] 12:25:36
Then you look at overall utilization. So if you have a certain product number of cloud credits, or if you have a certain allocation size, are you using these up because he spent some effort to get them, so you should use them, like so subscription.

[Enrico Fermi Institute] 12:25:52
Model, If If Google lets you use 10,000 cores for free as part of the subscription, it doesn't make sense to only 1,000.

[Enrico Fermi Institute] 12:26:00
There is no benefit on a penalty for using up your phone.

[Enrico Fermi Institute] 12:26:04
Yes, the other thing is turnaround time. So now run Time, I mean, is provisioning.

[Enrico Fermi Institute] 12:26:13
Turn around this is, comes especially into Hpc: If you talk about Lcf: if you talk about unit of provisioning what's associated with that is also.

[Enrico Fermi Institute] 12:26:26
The latency is very. We are used to that with our normal grid operations.

[Enrico Fermi Institute] 12:26:33
I'll see if you ask for a 1,000 notes. You can wait and you have no idea when you're gonna get it.

[Enrico Fermi Institute] 12:26:39
Eventually you'll get it It's not under your control.

[Enrico Fermi Institute] 12:26:44
and for all these metrics is how are we gathering them?

[Enrico Fermi Institute] 12:26:48
Okay, I mean our own resources. We have services in place. We have many years of preparation, so they have to see.

[Enrico Fermi Institute] 12:26:56
Just get them to forward information, to collect it. Hbc.

[Enrico Fermi Institute] 12:27:00
Cloud is different, especially Hbc. On the cloud. You can run whatever.

[Enrico Fermi Institute] 12:27:04
But Hbc is problematic. You need to collect statistics from the best use from the job system and so on.

[Enrico Fermi Institute] 12:27:13
I'll be how you forward it. So it's actually collected in the right place, and you can compare it.

[Enrico Fermi Institute] 12:27:19
You have phones. Texas? We got a question. Yep.

[Enrico Fermi Institute] 12:27:24
yes.

[Ian Fisk] 12:27:25
yeah, I had a question which was about the concept of You have nothing about Gpu efficiency.

[Ian Fisk] 12:27:31
If you have nothing but deep efficiency, it's just you haven't asked the the the Gpus themselves monitor.

[Ian Fisk] 12:27:38
Their utilization. Very well. Command is Nvidia Smi. They will tell you how much the memory and how much the theoretical processing capacity you're using

[Enrico Fermi Institute] 12:27:49
I'm nuts. I'm not saying that is nothing that you can run on a Gpu to tell you to what degree it's utilized.

[Enrico Fermi Institute] 12:27:56
What I'm saying is, I don't think we have any tool where, with our Gpu workforce, where we actually record this information, and keep track of it.

[Ian Fisk] 12:27:56
Okay.

[Ian Fisk] 12:28:06
Okay. But but at this point, in the same way that you that you record the Cpu efficiency with top you should. Simply There are tools that do exactly the same thing for Gpu

[Enrico Fermi Institute] 12:28:18
And

[Enrico Fermi Institute] 12:28:18
Yes, it just needs to go. Put it place and put into demonic traces, and it's it's more an indicator of how early we are in terms of adoption of gpu workflows in the experiments.

[Ian Fisk] 12:28:19
Okay.

[Enrico Fermi Institute] 12:28:33
Then It is an indicator of the lack of low level 2.

[Enrico Fermi Institute] 12:28:36
So all of the tools are there. It's just a matter of spreading it all through.

[Ian Fisk] 12:28:39
Yeah, I just this: This is the like, I can tell you.

[Ian Fisk] 12:28:43
I send a lot of email a week about people who are not using the Gpus especially.

[Ian Fisk] 12:28:47
Well, and so I It's probably something that should go early on It's a monitoring system, because it it.

[Ian Fisk] 12:28:55
It's not like it's not it's not.

[Ian Fisk] 12:28:56
It's hard to get

[Enrico Fermi Institute] 12:28:58
One thing

[Enrico Fermi Institute] 12:28:59
One thing conceptually, that that is not quite as mature as you know what these different things mean to be.

[Enrico Fermi Institute] 12:29:07
You're comparing cross sites.

[Enrico Fermi Institute] 12:29:12
How do you compare 1080 versus a a 100, or what it being maybe that example, you just round the the 1080 down to 0 have the problem solve.

[Enrico Fermi Institute] 12:29:22
But but trying to aggregate and cross. Compare to, could it?

[Enrico Fermi Institute] 12:29:29
I mean eventually one of the things you're asking here is if I shooting my money's worth.

[Enrico Fermi Institute] 12:29:36
And to be asked in many different directions, including from the from the site.

[Enrico Fermi Institute] 12:29:41
I wanted to start. Do that. You start to accounting, and it's these sorts of things aren't as accepted as what

[Ian Fisk] 12:29:45
right, but I

[Ian Fisk] 12:29:52
Right, but they're like one of the reasons why we have Hs O.

[Ian Fisk] 12:29:57
6 was that we had a variety of cpus weren't sure what the performance was going to be between them and this in Benchmark and figure out the relative capacity each of the sites, and it's not intrinsically more difficult than that and just there there's a wider there's a much

[Ian Fisk] 12:30:12
wider variation in the performance of Gpus

[Enrico Fermi Institute] 12:30:17
We need a Hso. 6 for Gpus. Maybe

[Ian Fisk] 12:30:20
Maybe if you need just 26. But yeah.

[Enrico Fermi Institute] 12:30:25
Another thing is, once you collect this, it's it's not good to me that we know, I mean.

[Enrico Fermi Institute] 12:30:29
And Cpu efficiency. We kind of have an idea of what's bad I mean.

[Enrico Fermi Institute] 12:30:34
Usually it's bad when we get dinked by the review balls that look at our Cpu efficiency and tell us we you make bad use of the resources for you It's It's been. Unclear to me on the gpu side what is bad day I would say we don't have any clue

[Enrico Fermi Institute] 12:30:50
on the Cpu side, but we pretend we do. We have about the same amount of clue on We have even less, because from you know, architecture generation to architecture, generations.

[Ian Fisk] 12:30:59
right.

[Enrico Fermi Institute] 12:31:05
Of Gpus. Things are changing pretty wildly, and the performance basis is very, very different, for you know, a turing class chip versus Okay, So the conclusion is we need to learn to pretend to know what we're doing exactly we need to come up with, sufficiently.

[Enrico Fermi Institute] 12:31:22
Obsceneated language, so that we can sound like we know what we're talking.

[Enrico Fermi Institute] 12:31:25
About; the fact that it's 2,022 in our review boards.

[Enrico Fermi Institute] 12:31:31
Don't understand hyperthread. No, that suggests that 2,040, maybe we'll have Okay.

[Enrico Fermi Institute] 12:31:46
So I guess, or sorry. The 1 one thing I wanted to say is that I I, in terms of like even coming up with performance, benchmarks.

[Enrico Fermi Institute] 12:31:55
I don't know if it makes a lot of sense to compare how you're doing with a bunch of tennis versus how you're doing with a bunch of amperes or a a 100, or whatever, just because the way those peopleors work, is so different yeah that does

[Enrico Fermi Institute] 12:32:10
cut my point. Sorry Sorry we agree. Great! That's awesome.

[Enrico Fermi Institute] 12:32:15
you don't just tell me this next slide, but particularly when it comes to acquisition at some point, you have to feedback to the the powers that be how you spend the money whether it was funny money or real money and that starts to get into things like pledging and you know actually having

[Enrico Fermi Institute] 12:32:35
these resources effectively, more effectively, acknowledgment experiments.

[Enrico Fermi Institute] 12:32:42
Or we're gonna touch that third rail today, or like this is, there's an accounting, pledging discussion on on Wednesday morning, and part of benchmarking is part of that, because, yeah, as you said, the thing is this is Hbc as long as it's opportunistic.

[Enrico Fermi Institute] 12:32:57
no one cares. It's free resources. When people actually start pledging it.

[Enrico Fermi Institute] 12:33:02
Then then you get into comparing performance numbers, and are you meeting your pledge or you're not meeting your pledge, And then things like measuring these things correctly, or at least in the measuring them in the way that you come up with a defensible number Okay, but good one to 4 one interesting

[Enrico Fermi Institute] 12:33:23
thing I observe on the Wcg. Side this month is, especially as some some sites in Europe are saying that we, you know we can't keep.

[Enrico Fermi Institute] 12:33:33
The use running over the winter but we'd like to send, you know, have the same number of hours delivered, which, of course, courses not what we pledge on So I think there's gonna be more for interest and examining some alternate models where interest wasn't

[Enrico Fermi Institute] 12:33:51
before, But I I think we we really need to push.

[Enrico Fermi Institute] 12:33:56
having things like Hpcs. Quote a quote cap and not right right now.

[Enrico Fermi Institute] 12:34:03
The the value delivered officially to the experiments is rounded to 0, even though we know from the resource graphs, these are been delivered lots and months of in terms of events at that that's gonna break some point, and the fact that some of the traditional wcg

[Enrico Fermi Institute] 12:34:25
sites, or also hit the brakes on the old pledge Models might be as close as you could do in turn turn it into an option Yeah, The 2,021 see it the Crsg: and that the comp basically where we added, up what we actually delivered 2021.

[Enrico Fermi Institute] 12:34:44
Us Hbc was slightly above formula. Now, okay, there's the normalization.

[Enrico Fermi Institute] 12:34:49
Factors have larger Arabs. So it was basically comparable.

[Enrico Fermi Institute] 12:34:53
But but again right now at view from some angle, you're you're saying.

[Enrico Fermi Institute] 12:35:01
We delivered as much as Fermi lab did, but then the value was written to, 0 and because none of it quote official accounts.

[Enrico Fermi Institute] 12:35:09
And that's that's a problem. Now when the problem gets increased by bigger, Yeah, the question about the turnaround.

[Enrico Fermi Institute] 12:35:16
So the turnaround time with some set Hpc centers, allow you to reservations Where you plan ahead.

[Enrico Fermi Institute] 12:35:26
Does that change, Then some of these metrics, and then also the also the Is it also simple?

[Enrico Fermi Institute] 12:35:35
Simplify operationally the I can just tell you the experience we had.

[Enrico Fermi Institute] 12:35:44
We have used reservations for Cms and that's mainly for the reason that the type of work for we're sending does always work so we don't really care when it runs.

[Enrico Fermi Institute] 12:35:57
The I know that some of the neutrinos and science experiments, where they had a big specific production that you targeted in Hbc that they had scheduled They planned ahead makes perfect sense to do A reservation in that Scenario for us I don't see that it.

[Enrico Fermi Institute] 12:36:14
Would help as much, because the the turnaround time is not so much a problem in terms of

[Enrico Fermi Institute] 12:36:24
Basically not being able to plan work because most of our work we don't care if it runs this week or next week, or a couple of weeks later.

[Enrico Fermi Institute] 12:36:31
I mean, there's higher. There's high priority stuff, but we usually do it at soon, and then play with the prioritization.

[Enrico Fermi Institute] 12:36:37
The turnaround time is actually more an issue for us in our software stack, because the system is just not designed with with like a 2 week provisioning time, and all week provisioning the assumptions in there So this is more software.

[Enrico Fermi Institute] 12:36:53
Problem than then an actual work plan planning problem, movement.

[Enrico Fermi Institute] 12:37:00
It's still a useful metric to have, because it doesn't right.

[Enrico Fermi Institute] 12:37:05
So I mean, you cannot. If you need a week, you cannot put high priority stuff.

[Enrico Fermi Institute] 12:37:08
There that's that's relatively small invitation, because most of our work is not high priority, No matter if these things had, that's a different issue.

[Enrico Fermi Institute] 12:37:19
But most of the work is just just get it done. We come back later on a month and check that everything is done

[Enrico Fermi Institute] 12:37:30
Okay.

[Enrico Fermi Institute] 12:37:34
Other comments or questions from Zoom: Yeah, yes, Okay, there any questions that we didn't that we should be asking

[Enrico Fermi Institute] 12:37:51
I mean, just to go hit the the dead horse again. I think the the accounting porting of resources has to be a a top level item

[Enrico Fermi Institute] 12:38:07
That's got to be appropriate, zoom. So not a particularly interesting technical topic, Tv.

[Enrico Fermi Institute] 12:38:18
But that's okay. I mean, we we did talk a lot about this in in context of Hpc: Were there any specific comments folks wanted to make about this on cloud

[Enrico Fermi Institute] 12:38:33
We save some of that for the discussion tomorrow

[simonecampana] 12:38:36
sorry? Can I ask a question? This is the morning I It's following up on what Brian said.

[Enrico Fermi Institute] 12:38:38
Yep.

[simonecampana] 12:38:43
I think it would be interesting. In fact, if those resources that today are a bit special, they might not be in the future, could be accounted properly, which means basically being reported back.

[Enrico Fermi Institute] 12:38:45
Okay.

[simonecampana] 12:38:57
Who's that official accounting tools we use? Do you understand that?

[simonecampana] 12:39:02
So is. The problem is, I didn't get from the discussion is the problem technica.

[simonecampana] 12:39:07
It's well understood how to do it, but someone has to do the work because the vergeical ways, for example, of integrating Hpcs.

[simonecampana] 12:39:16
if you use an engine like, head cloud them, you can put some of the intelligence there, and the report upstream.

[simonecampana] 12:39:24
Your accounting records. But if you have something like a direct integration of the Hpc.

[simonecampana] 12:39:29
With the workload management system of the experiment. Like, for example, at the way Atlas is doing, you, You don't have that gateway.

[simonecampana] 12:39:39
You don't have that service you need in practice, Banda, or the workload management system, whatever.

[simonecampana] 12:39:46
That is for report upstream. So I think it's a good idea to look into that.

[simonecampana] 12:39:51
Do you have view of how to do it?

[Enrico Fermi Institute] 12:40:01
In terms of the typical pieces, I'm not so so worried. All right.

[Enrico Fermi Institute] 12:40:04
We've We've done this several times across multiple generations of technology.

[Enrico Fermi Institute] 12:40:09
So we we not like. It's the first time we we had to do an integration like that in the last 5 years.

[Enrico Fermi Institute] 12:40:15
Again. My My my worry is if we come in and say, you know, Oak Ridge delivered a 100 million Cpu hours to Atlas.

[Enrico Fermi Institute] 12:40:29
you know. How does that get contributed as part of a delivered resource to to the experiment? Yeah, how do we form I can say this, then, Does that help meet The us's commitments to the W Lcg: because right?

[Enrico Fermi Institute] 12:40:46
Now, it's it's a very different saying that.

[Enrico Fermi Institute] 12:40:48
Okay, a resource with W. To Gw: And that, did you?

[Enrico Fermi Institute] 12:40:53
Making that count, and that the official we have a touch that in 2 decades versus the technical mechanism to get an integer for pointing.

[simonecampana] 12:40:59
okay.

[Enrico Fermi Institute] 12:41:04
Yeah, we seem to reinvent that over 5 years or so.

[simonecampana] 12:41:08
No, I see. So it's basically a question of policy you are making, which is a good one.

[Enrico Fermi Institute] 12:41:11
Got it. It's

[simonecampana] 12:41:14
has to do a bit with the what the experiment considers the pledge resource, and a lot of the spgaments. We're considering the pledge of source something that they can use to run any workflow in a transparent way So I think, as long as one goes in this

[simonecampana] 12:41:29
direction you will get the buy-in from an experiment.

[simonecampana] 12:41:33
Otherwise there might be some discussions to have

[Enrico Fermi Institute] 12:41:37
Oh, I think there has to be some discussion, because you're I don't think for any, with an exception, Maybe the cloud.

[Enrico Fermi Institute] 12:41:46
And even then landscape is a good counter example, where it probably can't run any so.

[Enrico Fermi Institute] 12:41:52
but but to say that yeah, the experiment site a 1 billion Cpu hours, or again, just making up numbers on overage is worth nothing, because we can't run everything on there, I think, is pretty short-sighted but it is a very important discussion and you know what I find is policies that are

[Enrico Fermi Institute] 12:42:14
older. It's tend to be harder to update the fact that we have a really dug into this in 20 years means it's it's gonna take some effort to come to a a place where everybody is happy and feel that they're concerns are heard.

[simonecampana] 12:42:34
yeah, I have. I I see your point. I think there are a lot of problem.

[simonecampana] 12:42:39
Is that there is a large spectrum right? There are Hpcs that can be used almost for everything which you can say sort of a pleasant source.

[simonecampana] 12:42:46
There are Hbc's that can be used to run one generator.

[Enrico Fermi Institute] 12:42:50
Okay.

[simonecampana] 12:42:51
It's a bit short sighted to say that those are like any other facility.

[simonecampana] 12:42:56
So, and because the spectrum is broad is difficult to.

[Enrico Fermi Institute] 12:42:59
Please.

[simonecampana] 12:42:59
I agree with you is something that

[Enrico Fermi Institute] 12:43:08
Going back to the acquiring for use specifically for Hpc: So right now, you you mentioned that for Hpc.

[Enrico Fermi Institute] 12:43:18
Every you know, there are a couple of different kinds of proposals, suspicion types, and if it's leadership class or or user facility, so these still require proposals which each it each the border each year, or and and so forth, and and to me it seems like you can't, say anything you

[Enrico Fermi Institute] 12:43:39
can't tie that to any sort of pledge situation.

[Enrico Fermi Institute] 12:43:43
If you've got a proposal that has to be proved by a bunch of outside scientific, you know, Committee right?

[Enrico Fermi Institute] 12:43:51
There was something we had in affirmative action This I'm not on this.

[Enrico Fermi Institute] 12:43:58
But something came up. Lis mentioned something that they were high-level discussions within the agencies about better support for data sciences.

[Enrico Fermi Institute] 12:44:08
that was one of the area of discussion She didn't say anything, about.

[Enrico Fermi Institute] 12:44:12
At least there's discussions I don't know if it's to what extent that will go anywhere.

[Enrico Fermi Institute] 12:44:19
The one thing that with this here is nice proposal.

[Enrico Fermi Institute] 12:44:24
They've started out asking the last couple of years about special needs, and like multi year planning horizon.

[Enrico Fermi Institute] 12:44:31
And it seems this year they kind of they already know what they're going to give us for the next 3 years It almost sounded like the feedback but we still have to write a proposal, but they ask us to write a simple proposal so on the ns, on the nsf side we're

[Enrico Fermi Institute] 12:44:47
starting, to see, not not at like the biggest Frontera type skills.

[Enrico Fermi Institute] 12:44:53
Nsf: Start to give allocations as part of the yeah crypt proposal.

[Enrico Fermi Institute] 12:45:02
So if you want a proposal like Uscms ops it.

[Enrico Fermi Institute] 12:45:05
It comes with the allegation as opposed to having Soma other, you know.

[Enrico Fermi Institute] 12:45:10
Peer, review Committee. They basically do this as double jeopardy that they could give you the money and then have somebody else make you unable to. To.

[Enrico Fermi Institute] 12:45:24
Okay, So that that's beginning to go into the system, but not at the Us.

[Enrico Fermi Institute] 12:45:33
Lhc: Ops: skills. So you know, there's there's more than discussion.

[Enrico Fermi Institute] 12:45:39
There's actually a couple of examples of doing this at at modest scale, but not that the the biggest one

[Enrico Fermi Institute] 12:45:51
Not across the finish line, but you know it's starting to show up in solicitations and things like this.

[Enrico Fermi Institute] 12:45:57
We have provided this feedback to the to the age funding agencies.

[Enrico Fermi Institute] 12:46:03
Before I think of the 2,019 child meeting and counting, discuss there.

[Enrico Fermi Institute] 12:46:10
This is, it's difficult to write it a generic of okay, you know, for collaboration that has some broad mix of workflows that competitive against a specific.

[Enrico Fermi Institute] 12:46:26
You know scientific. They get compared by scientific marriage, right?

[Enrico Fermi Institute] 12:46:31
And so, and they're looking for specific specific outcomes like, What did you discover on this machine?

[Enrico Fermi Institute] 12:46:39
Because you know, because we awarded you this this allocation, and that's difficult.

[Enrico Fermi Institute] 12:46:47
If you're saying, Ok, we ran, you know, just a you ran a generic mix of simulation experiment for 2 that at least, on the Nsf. Side.

[Enrico Fermi Institute] 12:47:03
This is where they are looking to tie this to? Yeah, but you know you get get the allocation as part of the Usf Us or Usms operations It's it's just not you know obviously Then again time for this proposal, and rounding and still scaling up I think this has

[Enrico Fermi Institute] 12:47:27
 to be addressed I don't know if this is in this book.

[Enrico Fermi Institute] 12:47:30
Prep process. You know, we need to have much dedicated to this, but this is an issue, I mean, this is

[Enrico Fermi Institute] 12:47:42
Spend a lot of time writing and writing proposals, and they get reviewed by by a committee.

[Enrico Fermi Institute] 12:47:47
And yeah, the Lcf proposals. Basically, you have to dress another.

[Enrico Fermi Institute] 12:47:52
You have to do. You have to. Basically, we actually tried that because we did 2 proposals this year. One was Gpu.

[Enrico Fermi Institute] 12:47:59
We construction on summit, and that was approved. Because if something new is something we haven't done before, so the other one we intentionally kept it.

[Enrico Fermi Institute] 12:48:12
We didn't dress it up. That was General Monte Carlo production on theta like.

[Enrico Fermi Institute] 12:48:17
Get us get some resources increase, do like 10% extra. Well, just stand up for the color production that was rejected.

[Enrico Fermi Institute] 12:48:25
And that's basically what I It was. I expected that because it's not exciting.

[Enrico Fermi Institute] 12:48:32
It's something you can do everywhere. They look at it, they say, Why are you on the Lcf.

[Enrico Fermi Institute] 12:48:37
You can do this, Okay, somewhere else And that's the tension with the with the pledged allocation where the pledge is supposed to be able to do everything. But and again. I think that's why it, has to be a major outcome.

[Enrico Fermi Institute] 12:48:49
Report. There is we We have to make the agencies realize experiments that the global collaborations are write down their contributions to $0 and 0 cents because they can get some of these.

[Enrico Fermi Institute] 12:49:06
You know, in a way that we can actually plan and get them to

[Enrico Fermi Institute] 12:49:14
And part of it's gonna be a shift on on the Wc.

[Enrico Fermi Institute] 12:49:17
2 side, I think. But I I think it. We also have to kind of throw some cold water and the agencies to make them real.

[Enrico Fermi Institute] 12:49:25
Wake up and sit up and realize. Oh, I'm not getting my my credit for the money, and because that, you know effectively, they're putting in money.

[Enrico Fermi Institute] 12:49:32
Have a getting no credit for the money, and he should be.

[Enrico Fermi Institute] 12:49:39
I'm mad about it, and but what we've talked about yeah, 5, 6 years, and not lunch Let's move on to the last slide before reporting future workflows.