HP Cost
[Eastern Time]
[Kenyi Paolo Hurtado Anampa] 11:30:50
yes. So the next section in this license about cloud costs, and just like yesterday they green.
[Kenyi Paolo Hurtado Anampa] 11:31:04
Here means this is one of the questions that we need the from the charge that we need to answer.
[Kenyi Paolo Hurtado Anampa] 11:31:08
And it's basically what is the total cost of operating commercial power, resources collaboration workflows.
[Kenyi Paolo Hurtado Anampa] 11:31:15
And this is mostly focused on production workforce for the charge, both for computer resources as well as the operational effort for the Lc.
[Kenyi Paolo Hurtado Anampa] 11:31:24
Around 3, and so with that we will start with, what's the experience software in the cost for Atlas and Cms? So for another, do you Wanna take over from here?
[Fernando Harald Barreiro Megino] 11:31:38
yes, so I mean the content of this light is mostly my opinion and experience.
[Fernando Harald Barreiro Megino] 11:31:47
and now just to note that with with this Atlas covid project there will be a dedicated Tc.
[Fernando Harald Barreiro Megino] 11:31:59
Also total cost of ownership board that will then study the costs detail.
[Fernando Harald Barreiro Megino] 11:32:08
so to explain a little bit the the cost model, so that you have in the cloud.
[Fernando Harald Barreiro Megino] 11:32:13
So for the computer are different levels of of the virtual machine.
[Fernando Harald Barreiro Megino] 11:32:20
So you have the reserved instances What you say basically I will.
[Fernando Harald Barreiro Megino] 11:32:24
I reserve for so many cpus for a year, and but that also means that you are stuck for the full year with those reserved instances, and there is no real elasticity.
[Fernando Harald Barreiro Megino] 11:32:38
Then there is the on demand which is the appetite for for the on-demand virtual machines, where, when you want to request, a virtual machine, and then, once you have it that way too much, for you for like forever and then the lower t of the on the month is the spot
[Fernando Harald Barreiro Megino] 11:32:58
where you will request a virtual machine, and then can get it, and you can also.
[Fernando Harald Barreiro Megino] 11:33:06
It can also be taken away from you whenever Google needs it.
[Fernando Harald Barreiro Megino] 11:33:09
For someone else, or it can be just their cute and put somewhere else if they need to do some optimization within their computing center.
[Fernando Harald Barreiro Megino] 11:33:24
Yeah, it's I think it's 37. So it's not.
[Fernando Harald Barreiro Megino] 11:33:31
It's nothing, but we can in practice work with.
[Fernando Harald Barreiro Megino] 11:33:34
So if you get the kill signal, you lost the whatever, so was running on the Vm.
[Fernando Harald Barreiro Megino] 11:33:40
Until then.
[Fernando Harald Barreiro Megino] 11:33:44
but the experience with Spot. It's quite good, in my opinion, because the in Google in particular also have only the preemptable vms which had the maximum lifetime of 24.
[Fernando Harald Barreiro Megino] 11:33:59
Hours and now they've stopped that model, and they've moved to Spot, and there you can have the the virtual machines for a long time.
[Fernando Harald Barreiro Megino] 11:34:07
and I don't see a significant amount of failed, wasted world clock time because of spot and got this.
[Fernando Harald Barreiro Megino] 11:34:16
Well, you will see it later, but it's like 60% cheaper than on the month.
[Fernando Harald Barreiro Megino] 11:34:22
Then for the storage you have also different categories.
[Fernando Harald Barreiro Megino] 11:34:27
There is the Standard number line call line, like the the time to access your files.
[Fernando Harald Barreiro Megino] 11:34:33
Is the same with all of them, with with the th.
[Fernando Harald Barreiro Megino] 11:34:38
The Then you go to the right. You need to keep the data on the storage for longer, So I think, Neil, and you need to keep it 30 days cold. Line.
[Fernando Harald Barreiro Megino] 11:34:51
I don't know how many days I'm so.
[Fernando Harald Barreiro Megino] 11:34:52
Also the the question that you get, the more you pay by the access.
[Fernando Harald Barreiro Megino] 11:34:58
So in practice for the storage, we are always using standard the standard class.
[Fernando Harald Barreiro Megino] 11:35:05
So that's a traditional course model. Now, there is a new cost model.
[Fernando Harald Barreiro Megino] 11:35:09
That is what we are using in, and it's this: Us public sector subscription agreement with, Google And they're basically, if you are university or or a lot from the public sector, you can negotiate a fixed price for your computing needs So agreement
[Fernando Harald Barreiro Megino] 11:35:31
for 10,000. See total cpus and 7 petabytes of storage, and it's for $600,000 for 15 months.
[Fernando Harald Barreiro Megino] 11:35:48
and you pay that amount, and you don't need to worry.
[Fernando Harald Barreiro Megino] 11:35:51
If then you have more egress or less ears to optimize your, you're agreement, obviously.
[Fernando Harald Barreiro Megino] 11:35:58
And this protects you from surprises Under the end of your 15 months.
[Fernando Harald Barreiro Megino] 11:36:04
I guess it will be a review like while you, your egress, is out of control to try tomorrow or not, or maybe they are happy with the situation, and they get renegotiated and I don't want to talk about the exact amount of dollars that we have in the in our agreement.
[Fernando Harald Barreiro Megino] 11:36:24
But I just want to say that it's very favorable, and it's lower than that this prices the under thing to consider in in these clouds is that the resources are very elastic.
[Fernando Harald Barreiro Megino] 11:36:41
So it's a bit what they try to show with in the task Example: the cost.
[Fernando Harald Barreiro Megino] 11:36:45
For 10,000. Cpus for and 1 h. It's the same as the cost of for one Cpu.
[Fernando Harald Barreiro Megino] 11:36:50
For of 10,000 h, so you can run the real estate without a major cost increase, and also, since it's the last like, if you ramp down, you don't need to keep any other resources, and you just really pay for what you use and from an operational perspective.
[Fernando Harald Barreiro Megino] 11:37:17
in in my opinion, it's very low cost. There is, I mean, the whole development setup on operation of the of the Lutheran part was done by one of the route experts and also all of the development setup and operation for the Underpassed by one panel expert fractions, of the of
[Fernando Harald Barreiro Megino] 11:37:38
the right, And I also think that here this model is like really pure devops for the most pure form of it.
[Fernando Harald Barreiro Megino] 11:37:49
You operate aside. You see, I'm you also learned things that are not good for the site that you can improve in in Fanda or in harvest.
[Fernando Harald Barreiro Megino] 11:37:59
And then you go and change those things, and also with the same amount of Fd.
[Fernando Harald Barreiro Megino] 11:38:08
Resources like, if I run a 10,000 core. Cluster, 30,000 core class that doesn't really make up difference.
[Fernando Harald Barreiro Megino] 11:38:18
I'm now moving to the plot on the right, so here what I'm showing is the all of the pins, except the last one to the right are simulations using the cost you later.
[Fernando Harald Barreiro Megino] 11:38:34
the first one are on Amazon, the second ones are on Google. Good.
[Fernando Harald Barreiro Megino] 11:38:38
and I didn't average of the Usda tools and play so that there are in average it's 10,000 data cpus.
[Fernando Harald Barreiro Megino] 11:38:48
Of course, 7 petabytes of of storage
[Fernando Harald Barreiro Megino] 11:38:55
I don't know what she keeps writing, and then also an average.
[Fernando Harald Barreiro Megino] 11:39:04
There was 1 point, 5 petabyte of egress per month.
[Fernando Harald Barreiro Megino] 11:39:06
I looked it up in the dtmashboard, and then I went to the Google price calculator and I using different types of oh Pms.
[Fernando Harald Barreiro Megino] 11:39:17
I calculated the cost. The blue part is the the cpu.
[Fernando Harald Barreiro Megino] 11:39:21
The red part is the storage. 7 petabytes, and the yellow part is the 1.5 kB of egress per month, and then well so depending on what the type of compute you use you pay you can reduce it so this is the first one is on
[Fernando Harald Barreiro Megino] 11:39:38
the demand. Second one is if you pay one tier upfront on Amazon, then if you reserve for a year on Amazon.
[Fernando Harald Barreiro Megino] 11:39:46
But you don't pay upfront, and it's a little bit more expensive.
[Fernando Harald Barreiro Megino] 11:39:49
And then you reserve for 3 years, and then you see that the price starts dropping considerably, and the last one for Amazon is the Amazon spot, and you see that the the Cpu part is really much lower than the which is the Amazon on demand.
[Fernando Harald Barreiro Megino] 11:40:08
Then, if we move to the Google part, Google is a little bit, She preferred the Cpu in at least for for the calculations that I did for the egress and the storage, is more or less the same, as amazon and then the very last pin that I took on
[Fernando Harald Barreiro Megino] 11:40:30
the billing report of of the Google Cloud console for the last 30 days under extracted how much we have been spending on each one of the things and to compare it with what I had done in my theoretical calculations the Cpu was a little bit cheaper So
[Fernando Harald Barreiro Megino] 11:40:50
we use Spot. So you have to compare it with the Gcp.
[Fernando Harald Barreiro Megino] 11:40:53
Spot. It's a little bit cheaper. Also I didn't use the full 10,000 cpus, but only in my 2 9,200 for the story to It's much cheaper than the others, but also we don't have the 7 petabytes. Of data.
[Fernando Harald Barreiro Megino] 11:41:12
Yet We have only 1.6 bit byte. So that's that explains it.
[Fernando Harald Barreiro Megino] 11:41:16
And then egress. We are at. We did 1.2 Pedros of E address, according to the Tcp Billing, which is very close to what I had gotten in my models?
[Fernando Harald Barreiro Megino] 11:41:28
and so that's what you would be paying if you would pay list prices.
[Fernando Harald Barreiro Megino] 11:41:33
But again in our user agreement, it's the the what we would keep paying effectively. It's it's lower than that on the this is it for this night.
[Fernando Harald Barreiro Megino] 11:41:46
I see there are Yeah.
[Paolo Calafiura (he)] 11:41:52
quick, quick, question actually It's a call man by the question.
[Paolo Calafiura (he)] 11:41:57
the subscription price is, of course, very advantageous, but it does.
[Paolo Calafiura (he)] 11:42:03
It does kind of remove the elasticity you mentioned, because you know, if you use one Cpu for 10,000 h, you are not using your subscription very well at all, so that that that, was the only comment that I wanted to make and then I and follow our question is anyone on top with with
[Fernando Harald Barreiro Megino] 11:42:16
Hmm.
[Paolo Calafiura (he)] 11:42:22
Amazon about a model similar to this Google subscription?
[Fernando Harald Barreiro Megino] 11:42:28
So about elasticity, not completely because the agreement is for 10,000 little cpus in average, so you could be using one month, 5,000 on the next month, 15,000 But yeah.
[Fernando Harald Barreiro Megino] 11:42:46
If you arrive on the last day, and want to use your average 10,000 digital Cpu: so 15 months on the last day, that will be very difficult.
[Fernando Harald Barreiro Megino] 11:42:57
But the zoom, with your your resources, So there's some elasticity there is, and about the Amazon question, I I don't.
[Paolo Calafiura (he)] 11:42:57
yeah, I did.
[Kaushik De] 11:43:19
yeah, we we. We have not had the conversation with Amazon with Amazon.
[Kaushik De] 11:43:24
We only use credits in the old traditional way.
[Kaushik De] 11:43:33
So in some sense it's good because we have the side by side comparison with Amazon via I'll set them out to fix credit.
[Chris Hollowell] 11:43:56
I yes, it. You know from my experience, the the a lot of the Cloud providers.
[Chris Hollowell] 11:44:01
They're not really guaranteeing a specific Cpu model.
[Chris Hollowell] 11:44:05
It's sort of nebulous what cpu they're they provide.
[Chris Hollowell] 11:44:09
So I mean, I guess the question is, you know, so you have 10,000 chorus.
[Fernando Harald Barreiro Megino] 11:44:21
they do not, they do not tell you exactly what's the Cpu model, but some family.
[Fernando Harald Barreiro Megino] 11:44:31
So, for example, I used the N. 2, and that is cause Kate Link or Ice Lake, I think, would I'm not the Cpu expert but th th those are for Google the newer generations.
[Fernando Harald Barreiro Megino] 11:44:44
And if you take the n one you go to the order generations.
[Fernando Harald Barreiro Megino] 11:44:46
So, yeah, it's you. You don't. Yeah, you're you're more or less right.
[Fernando Harald Barreiro Megino] 11:44:55
That don't know exactly what the Cpu. That's an approximation
[Steven Timm] 11:44:59
Oh, you think hmm
[Enrico Fermi Institute] 11:45:00
Do they not expose anything into the in the Os
[Enrico Fermi Institute] 11:45:05
Okay.
[Steven Timm] 11:45:06
So it Yeah, I had my student actually run this house for most of the new Google instances this summer.
[Steven Timm] 11:45:18
I have the numbers, we we we've got most of the Google aspects available.
[Steven Timm] 11:45:22
I think we want some
[Fernando Harald Barreiro Megino] 11:45:24
I would be interested in having that
[Chris Hollowell] 11:45:26
right, right.
[Steven Timm] 11:45:37
okay.
[Chris Hollowell] 11:45:39
I guess the issue, there though is, since they're not guaranteed any Cpu model.
[Chris Hollowell] 11:45:44
In particular, there would change
[Enrico Fermi Institute] 11:45:57
Here we had a comment from Dirk
[Dirk] 11:46:04
one was about the elasticity which was already covered, so it seems to be possible within within limits.
[Dirk] 11:46:12
But you probably, if you have a 10,000 average, you can't run 120,000 like one month, and nothing the rest of the year.
[Enrico Fermi Institute] 11:46:16
Thanks.
[Dirk] 11:46:18
That's probably not gonna fly. Let's see.
[Dirk] 11:46:22
Seems to me but the the other one was, I think, I mean, we talked about these pricing plots a a a bit.
[Dirk] 11:46:28
I think I finally I think, understood what that last bomb means.
[Dirk] 11:46:35
So that's from the within the subscription. That's from the running counter insight.
[Dirk] 11:46:40
So it's in some sense fake pricing, right?
[Dirk] 11:46:42
It's because you you pay the subscription price, but they still tabulate what things cost?
[Fernando Harald Barreiro Megino] 11:46:48
yeah. So with the subscription, what? And they are doing is all the time filling up our credit.
[Dirk] 11:47:01
Quote unquote.
[Dirk] 11:47:01
Okay.
[Ian Fisk] 11:47:11
Yup, my! It was a question actually which was It is as interesting as to see the various models between the 2 great cloud providers.
[Ian Fisk] 11:47:19
Has anyone done an updated what it's actually costing us to host these things?
[Ian Fisk] 11:47:24
Because I'm looking at these numbers, and I I know the size of the facility that we run versus the cost of the hosting and the operations. And these numbers are dramatically higher than we're paying
[Ian Fisk] 11:47:44
So I'm putting in that right bye, bye, I
[Ian Fisk] 11:47:49
Okay, I am, including in that price the cost of the hosting.
[Ian Fisk] 11:47:53
So what it costs to rent the space, to power, the machines to buy the machines, to operate, the machines, to administer the machines and support people, using the machines
[Fernando Harald Barreiro Megino] 11:48:07
but then everything that is installed on top is, it's not included this.
[Ian Fisk] 11:48:11
Oh, really! What what, What do you mean? What's installed on?
[Fernando Harald Barreiro Megino] 11:48:16
All the services that you are running
[Ian Fisk] 11:48:17
I am, including all like what I mean, services like the back system, and the oh, putting, including all of that, too, putting all of those things
[Ian Fisk] 11:48:28
So I'm including including the 15 person staff that we run the place, who add, plus the cost of the hosting, the facilities plus the cost of operating the storage, external networking, etc.
[Fernando Harald Barreiro Megino] 11:48:40
I mean the list prices in particular. If you go on the month.
[Fernando Harald Barreiro Megino] 11:48:45
so I've been told not to compare. I I I compared it myself with a Usda tool, and if you use on demand instances, it they considerably higher our subscription agreement is very similar to our usda 2 without saying How much we used to costs.
[Ian Fisk] 11:49:01
Right.
[Fernando Harald Barreiro Megino] 11:49:17
because it Oh, the one because because it creates a conflicts and fights.
[Enrico Fermi Institute] 11:49:26
no, but as a quality of service for the the If you're going to compare it to a tier, 2, right is the quality of service that you have to provide for the storage.
[Enrico Fermi Institute] 11:49:38
The same on Google Cloud as it is for midwest tier, 2, for for example.
[Fernando Harald Barreiro Megino] 11:49:45
I mean
[Enrico Fermi Institute] 11:49:46
Because that has an operate that has an operational effect
[Fernando Harald Barreiro Megino] 11:49:51
I mean. My opinion is that the quality of service in Google is something that no, I mean, they simply have thousands of.
[Enrico Fermi Institute] 11:50:02
No, I mean the Atlas service, the Atlas services that are running the Rc.
[Fernando Harald Barreiro Megino] 11:50:02
So the quality of yeah.
[Enrico Fermi Institute] 11:50:08
That runs at Google right? Can it, for instance, be a nucleus so that it can serve out data and stuff like that?
[Enrico Fermi Institute] 11:50:13
That's what I mean by quality of service not their underlying layer.
[Enrico Fermi Institute] 11:50:16
That's that's all good. What I really mean is the Lwlcg layer of services and code that have to run on it to to have it.
[Fernando Harald Barreiro Megino] 11:50:16
So
[Enrico Fermi Institute] 11:50:27
Behave like a typical tier. 2 grid site
[Fernando Harald Barreiro Megino] 11:50:30
I I've been working on the panel site, and that works as good as any It was tier, one or room I mean.
[Fernando Harald Barreiro Megino] 11:50:40
It's got to know if better, because I don't look that much.
[Fernando Harald Barreiro Megino] 11:50:43
I did it. Other sites, but it's completely flat, so there is never wasted cause.
[Fernando Harald Barreiro Megino] 11:50:49
The failure rate is very, very low, and when there is a value rate it's usually done.
[Fernando Harald Barreiro Megino] 11:50:57
It's usually cost to by misconfiguration. Just that.
[Fernando Harald Barreiro Megino] 11:51:01
I don't have the. It's new. We're running it since a month.
[Fernando Harald Barreiro Megino] 11:51:07
And, for example, I underestimated the disk. Things like that this and half a year in my the pandemic will run as good or pay.
[Ian Fisk] 11:51:21
I I guess I do. I would I just go back to my point, which I think is important for the report to follow is that we're always in a situation where we're making a choice in terms of how we allocate the resources and we're always having to cut back on something else to afford so
[Ian Fisk] 11:51:33
that so in in some sense, we're at some point, we're gonna have to make an argument that says, using the cloud is less expensive by so metric.
[Enrico Fermi Institute] 11:51:53
Seekashka has had his hand raised for a while to jump in
[Kaushik De] 11:51:57
yeah. So I wanted to address 2 of the points that we have had extensive discussion on.
[Kaushik De] 11:52:07
One is the elasticity doesn't seem to care.
[Kaushik De] 11:52:11
They're perfectly fine. If you want to use 100,000 cores for one month instead of 10,000 cores, how the duration of the project we are planning to test that when we moved the later parts of our planned program of work And are in these studies with Google But we certainly
[Kaushik De] 11:52:30
plan to test both models. The only reason why we started with the flat model is because that's what our current computing systems are designed for, And we wanted to give that a quick. Test.
[Enrico Fermi Institute] 11:52:39
Okay.
[Kaushik De] 11:52:48
We don't have to continue this way. We could not run anything for 3 months, and then we could run 5 times higher for a month.
[Kaushik De] 11:52:56
It's it's completely elastic up to the the limits of the resources of the data center.
[Kaushik De] 11:53:03
And then, of course, one can scale up by going to multiple data centers.
[Kaushik De] 11:53:06
So that's the elasticity issue. Even with the subscription model, because we have discussed it with them.
[Kaushik De] 11:53:14
I think the cost, comparison issue. I think it's an important one, but I think we have to be a little bit careful, because we will never come to a conclusion if we ask Google or the team that's using Google to come up with the cost off.
[Kaushik De] 11:53:31
A T. I wanna tear to side, I mean, that's just will never work.
[Kaushik De] 11:53:35
You know that it will never work, because every time somebody from the outside tries to value it the cost of.
[Enrico Fermi Institute] 11:53:38
Me.
[Kaushik De] 11:53:42
Do you want N. 2 to decide? It will be. There will be something that people will find to say that it was not done correctly.
[Kaushik De] 11:53:53
So. I think it's that you're one, and 2 sites who actually truly have to do the costing, and they actually have to do the comparison.
[Kaushik De] 11:54:03
And they actually have to decide what is best for them to have on prem resources or offering resources.
[Kaushik De] 11:54:09
And in what particular combination do they want to do it?
[Kaushik De] 11:54:12
I think it's up to the tier. One; and tier 2 sites.
[Kaushik De] 11:54:16
It's not up to the people who Alright, using Google and Amazon And it's certainly not up to the salespeople from Google and Amazon to tell us how to they can do it cheaper all.
[Kaushik De] 11:54:25
We can do. And I think that's what we are focused on doing.
[Kaushik De] 11:54:28
And I think that's really last video updates, 2 plots.
[Kaushik De] 11:54:32
Here is what is the cost of doing this and that on Google and Amazon.
[Kaushik De] 11:54:39
And I think that's how we make progress. Is we alright at as transparent as possible, as many different kinds of tests as possible.
[Kaushik De] 11:54:48
Explore all the possibilities that we can do.
[Kaushik De] 11:54:54
And then we an experimentalist. We do that through this project, over the next 15 months, and then we provide that information Then it is up to tier one tier.
[Kaushik De] 11:55:05
2, since people of various kind to come and argue this way and that way, and I don't think we should be as a technical thing, we should be part of that
[Enrico Fermi Institute] 11:55:15
No; but you have the caution. Yeah, if hold them the lost capabilities.
[Enrico Fermi Institute] 11:55:23
And so I'll use an example as a lot of the engineering was pulled out of the physics departments and went to the National Labs University groups lost capabilities.
[Enrico Fermi Institute] 11:55:34
They couldn't do certain things on detector projects. This will be the same.
[Enrico Fermi Institute] 11:55:39
This we have to quantify that effect? If you were to, for instance, move all the compute to the clown, what would we lose
[Kaushik De] 11:55:46
I complete. I completely agree with you, but those are not part of it.
[Kaushik De] 11:55:50
Technical study of what we can do on Google and Amazon.
[Kaushik De] 11:55:54
Those are really discussions within the field of how we move our field forward.
[Kaushik De] 11:55:59
I I think we should separate the 2. I don't think we should mix up the 2.
[Kaushik De] 11:56:02
I think we should look at the quality of service. I think we should look at the type of service.
[Kaushik De] 11:56:07
I think we should look at the services that are actually global and and and provide that we look at the past.
[Kaushik De] 11:56:17
That's the scope of what we're doing beyond that.
[Kaushik De] 11:56:21
Of course, is up to the field to decide
[Enrico Fermi Institute] 11:56:24
But even in the technical cost thing you, because of the hour, provided labor to it.
[Enrico Fermi Institute] 11:56:29
Don't we also have to capture good? This? The labor needed to to have the same quality of service as same from the experiment, I'd say typical tier 2
[Eric Lancon] 11:56:51
yes, so I wanted to come back on a few statements which were made, I think we're we need to be very careful about the general statement, like it's cheaper than a tier 2 this state those statement.
[Eric Lancon] 11:57:15
Do not represent us at last; and as you should be indicated on the slides, if they are such that statements there, there is a working group with and Atlas which is being set up to supposedly look at the Tco for operating on the Cloud and the Tier 2 so may want.
[Eric Lancon] 11:57:33
to to wait for the conclusion of this working group. What I would like to say is that that Will Kevin?
[Eric Lancon] 11:57:42
Well, very well aware of the cost of cloud compared to on-site operation, because for any big investment we platform comparison of the cost.
[Eric Lancon] 11:57:55
Okay, the on the cloud, including the feminist Google, Do we discount which is being used by Atlas?
[Eric Lancon] 11:58:05
And we have, as it was noticed by Yan Fisk.
[Eric Lancon] 11:58:12
the cost, really prohibitive. I cannot give you exact numbers, because we cannot provide the touches cost.
[Eric Lancon] 11:58:23
But you know really much slower than any solution which is available on the cloud.
[Fernando Harald Barreiro Megino] 11:58:43
regarding your first comment. No one or I didn't hear anyone saying that this is cheaper than Usdo.
[Fernando Harald Barreiro Megino] 11:58:52
I don't know where you got. No, I said, It gets similar with the with the user subscription
[Enrico Fermi Institute] 11:59:01
Okay.
[Fernando Harald Barreiro Megino] 11:59:05
Well, okay. In in any case, explicitly, I didn't put a usual cost, and for the Tcr.
[Fernando Harald Barreiro Megino] 11:59:13
it's what I said at the very beginning, but the Tc.
[Paolo Calafiura (he)] 11:59:27
yeah, I won't. I won't want to want the what. Oh, sorry me.
[Paolo Calafiura (he)] 11:59:35
I didn't see the way ends. Can I go
[Paolo Calafiura (he)] 11:59:39
I apologize. So then I want to make a comment, which is that one thing we have to keep in mind is this derivative?
[Paolo Calafiura (he)] 11:59:50
And so the the costs, the comparison with cost of proud to do on the resources, was just 3 close The first time we did it, which was about 2,016.
[Paolo Calafiura (he)] 12:00:04
I mean it was like order of money. It's more expensive.
[Paolo Calafiura (he)] 12:00:07
And while I agree with Eric, that is, that is not yet done at the cost comparison.
[Paolo Calafiura (he)] 12:00:14
And now it's actually it's actually what to do in the cost comparison, and probably it will come.
[Paolo Calafiura (he)] 12:00:20
It will come still more expensive On the cloud side than than the own side, but not by a factor of 10.
[Paolo Calafiura (he)] 12:00:27
So I think one of the important roles of these investigations is to be ready to be ready in case, for some reason, Google Cloud or aws, they, they can buy cpu and storage at prices We Don't.
[Paolo Calafiura (he)] 12:00:44
Have access to So let's not just see it, as is a short term effort.
[Paolo Calafiura (he)] 12:00:52
But there's an effort which is thinking about what's gonna happen in 5 years.
[Eric Lancon] 12:00:55
no, I agree. I agree. I agree, Paula. We should keep a close eye on the cost, and if the services for for equivalent level of service cheaper on the cloud, we should consider going to cloud solution for some of the application.
[Steven Timm] 12:01:25
So we have a couple of comments. One is that go 2.
[Steven Timm] 12:01:29
3 years ago, just before Covid. It was a very big study done. It's for me.
[Steven Timm] 12:01:35
I tried to. What would it cost to run the Ruben Data Center here as opposed to ringing on Cloud and tried to.
[Steven Timm] 12:01:42
And we tried to cost that all. Okay, do not exactly know all the numbers there, but there was a very comprehensive study that was done, and that's sort of that's A data point.
[Steven Timm] 12:01:51
Yeah, would I could. So we must be familiar with it already. New fees could probably get there for you.
[Steven Timm] 12:02:00
If anybody wants to add, let's come in here
[Ian Fisk] 12:02:01
bye, I think that that those numbers are public, and people want to see them.
[Steven Timm] 12:02:05
Okay? Yeah, huh? Right
[Steven Timm] 12:02:13
Great. No, no! The other thing that we've noticed from 6 years ago, when we first did the big Cms.
[Steven Timm] 12:02:21
Demo on Amazon until now is that probably by pricing is gone up by factor 2.
[Steven Timm] 12:02:26
Hi! Amazon, so you could in 2,016 you could go a 25% amount of your price.
[Steven Timm] 12:02:33
You can't do that anymore and get any cycles the th that's of interest, I think.
[Steven Timm] 12:02:38
And then the third thing is that as far as costing, what does it cost to run it here to on the cloud as opposed to web through our many estimates of that on the website some would agree by factor 4, we're always going to be difference in but sooner or later
[Steven Timm] 12:02:59
we're going to go to. Do we say we need more money to put more computer.
[Steven Timm] 12:03:03
We need more money for another building. And we're not gonna get it. So there will be a limit to how much we can put on a site.
[Steven Timm] 12:03:10
And then that may be the driver eventually to why we need to go to the cloud eventually.
[Enrico Fermi Institute] 12:03:22
Okay, thanks, Keith. We'll go to Tony.
[Fernando Harald Barreiro Megino] 12:03:34
This is 10,000. This is using the cost, I mean do all of the bus except the last one.
[Fernando Harald Barreiro Megino] 12:03:41
It's 10,000 little cpus where you run.
[Fernando Harald Barreiro Megino] 12:03:44
Whatever you want, 7 petabytes of standard objects store, and 1.5 petabytes of egos per month.
[Fernando Harald Barreiro Megino] 12:03:52
It's not it's support, whatever you're using, for it's not strictly related to a simulation
[Enrico Fermi Institute] 12:04:03
Ask a question. Go ahead. You have the the hoax in place, Amanda.
[Enrico Fermi Institute] 12:04:09
Capture, What Cpu model the job reports cause.
[Enrico Fermi Institute] 12:04:16
Then we can turn around and figure out for the number of virtual cpus you've used for some period of time.
[Enrico Fermi Institute] 12:04:22
But the hep specho. Sex equivalent is, and then then one can compare it, at least in you know we know what the tier two's provide in terms of Hs. O.
[Enrico Fermi Institute] 12:04:36
6,
[Fernando Harald Barreiro Megino] 12:04:38
So I I I have not looked into that for many, for most of the great sites it gets reported back as the okay.
[Fernando Harald Barreiro Megino] 12:04:50
Yeah, like the pilot looks for the for that information and reports it back.
[Enrico Fermi Institute] 12:04:52
Okay.
[Enrico Fermi Institute] 12:05:01
Then it might be very interesting to compare that to the benchmark jobs, where you know I should take with enough spread, so that you get the distribution of what they're actually giving you Yeah, because at the end of the day right we get paid in the us dollars per H. S. O.
[Enrico Fermi Institute] 12:05:23
6, okay.
[Enrico Fermi Institute] 12:05:31
Comment from Ian
[Ian Fisk] 12:05:34
I thought Stephen was done, but yeah, I guess But I just wanted to go back to this bel labor.
[Ian Fisk] 12:05:44
This point about cost, and I think the like. I think one thing that we need to assess as a field is to what are the economics that changed as every time we do this evaluation we find that things are a little bit closer to being competitive and at some point maybe they will make the transition over
[Ian Fisk] 12:05:59
but something has to happen which is then basically either the the economy of scale associated with am aws and Google has to be so large.
[Ian Fisk] 12:06:09
But they can do it more or less, or work cheaper than we can, and still make money, And whether that's a location facility, or whether that's the fact, that we use all our resources only a fraction of the time, or whatever But something has to change, because at the end, of the day like for the
[Ian Fisk] 12:06:24
same reason I don't drive a rental car to work.
[Ian Fisk] 12:06:28
if you have a facility, if you have a facility which you're using all of the time was you operated yourself.
[Ian Fisk] 12:06:33
It's very hard for someone to undercut you, unless they can, either.
[Ian Fisk] 12:06:38
They're so large, or they're so cheap, or they bring in cheaper places something we must be able to identify the thing that is going to make it competitive.
[Gonzalo Merino] 12:06:58
yeah, so, just hi. Briefly a brief comment just wanted to subscribe to some a previous comment from Kaoshik And I must say I'm a little bit surprised about all this this discussion, whether it is cheaper, or more or more expensive I really think that the I mean has like 170 sites
[Gonzalo Merino] 12:07:17
so that the answer will totally be different. For each of those sites.
[Gonzalo Merino] 12:07:20
So I think they're going through a Gaussian, which I totally subscribe Is that the value in this exercise, or at least part of it is, Okay, we need to get these numbers like the fernando show so that's super useful.
[Gonzalo Merino] 12:07:31
So what's the cost of running this in the cloud in a commercial cloud?
[Gonzalo Merino] 12:07:35
And then is, is for each of those 170 sites to get this number, and then compare to their internal costing which completely, will change.
[Gonzalo Merino] 12:07:43
I mean depending on size, depending on country. The labor cost is factors.
[Gonzalo Merino] 12:07:49
Difference from different countries, So so I mean I I I don't think I mean discussing on whether it's more expensive or or more or cheaper than these on that, side I think it's useless, and whether it's Fermila or a Tier 2 here in Czechoslovakia
[Gonzalo Merino] 12:08:01
or in Spain. It's it's for each of the sites in every country to get this number compared to their cost, which everybody knows.
[Gonzalo Merino] 12:08:10
And then react accordingly. I would say that that's the value I see, and the and the example from the rental card.
[Enrico Fermi Institute] 12:08:37
shooting.
[Shigeki] 12:08:41
yeah. One comment I have is this: There, there's really no incentive for any of these cloud providers to become the lowest cost per provider They're in the business to make money, right?
[Shigeki] 12:08:54
And they have hordes of the accountants and supercomputers that are that are constantly hedging the cost of everything right there.
[Shigeki] 12:09:03
There, there, there business model is value. Add not not to drop to the lowest, lowest cost per provider.
[Shigeki] 12:09:11
Right.
[Dirk] 12:09:21
the power of competition, I mean they are competing against each other
[Dirk] 12:09:29
So
[Enrico Fermi Institute] 12:09:34
Yeah, yes.
[Dirk] 12:09:36
I mean, that's what the isn't. This saying in any mature market?
[Dirk] 12:09:39
The price of a service will basically go down so that the profit approach to 0
[Enrico Fermi Institute] 12:09:48
Have you flown recently That's a mature market, and the prices are going the other one.
[Dirk] 12:09:56
Well, big slash supply. So that's that's the thing about these data centers.
[Fernando Harald Barreiro Megino] 12:10:09
Okay, I think we can go then to the next slide with K.
[Fernando Harald Barreiro Megino] 12:10:16
You.
[Kenyi Paolo Hurtado Anampa] 12:10:19
okay. Yup
[Kenyi Paolo Hurtado Anampa] 12:10:28
We'll take that as now. So Okay, moving on on the Cmx experience.
[Kenyi Paolo Hurtado Anampa] 12:10:33
So I tried to summarized and boot. Have a few numbers there from those sources.
[Kenyi Paolo Hurtado Anampa] 12:10:40
This is from one paper and some slides of the world.
[Kenyi Paolo Hurtado Anampa] 12:10:45
That was done 5 6 years ago, when Amazon on Google Cloud.
[Kenyi Paolo Hurtado Anampa] 12:10:49
So again, this numbers are not upgrade, are from 2,016 to substance 17.
[Kenyi Paolo Hurtado Anampa] 12:10:57
So things have changed. But the conclusion, the high, level summer of the the conclusion there is that the cause of per core hour for both aws and Google Cloud were close to similar But then the work that was on an amazon was on over the cover of a few days about 8 days and you can see in
[Kenyi Paolo Hurtado Anampa] 12:11:20
the stop. Right. Plot the and th the green. That is the production on awws.
[Kenyi Paolo Hurtado Anampa] 12:11:29
Just in, kept out on the formula side, and then on the bottom block at It's what do you have from Google Cloud?
[Kenyi Paolo Hurtado Anampa] 12:11:42
And the work on Google cloud was done over the course of about 4 days.
[Kenyi Paolo Hurtado Anampa] 12:11:47
the goal was to double the size in terms of total available course.
[Kenyi Paolo Hurtado Anampa] 12:11:57
With respect to the what we had in the global pool for the demo, the demo was done using.
[Kenyi Paolo Hurtado Anampa] 12:12:04
Yeah, that production, simulation workflows and the on premises.
[Kenyi Paolo Hurtado Anampa] 12:12:11
Estimate that I put on the paper on the the slides are, what's the estimate on the paper? Again?
[Kenyi Paolo Hurtado Anampa] 12:12:18
You have the services linked from Archive. They're in this.
[Kenyi Paolo Hurtado Anampa] 12:12:25
and then the the other factor just focus on the operational airport.
[Kenyi Paolo Hurtado Anampa] 12:12:32
So for this I got you put from the team from Cloud and the completion is that there was initial effort mostly related to monitoring.
[Kenyi Paolo Hurtado Anampa] 12:12:47
And this was to prevent waste of computer resources to track It's stuck jobs or jobs getting or going to slow identify.
[Kenyi Paolo Hurtado Anampa] 12:12:59
I bones identify huge log files that call that the cutting current to high transfers.
[Kenyi Paolo Hurtado Anampa] 12:13:06
But then after that, the current maintenance is low in terms of effort, with an estimate of just one.
[Kenyi Paolo Hurtado Anampa] 12:13:17
If you for that, occasional for things like, for, for, for example, they th this it is still maintained up to up to today.
[Kenyi Paolo Hurtado Anampa] 12:13:27
For okay. Tms wants to use it again. And so I I feel a few months ago they didn't work on integrating support for Id Token so that you can, for example, for both main, number you can go.
[Kenyi Paolo Hurtado Anampa] 12:13:49
So alright. But we have the last slides with this basically strategy, considerations and discussions, and these are just some bullets to.
[Kenyi Paolo Hurtado Anampa] 12:14:01
We talked a lot about cloud costs already, and there are some other bullets there right related to the egress. Cost what is the role?
[Kenyi Paolo Hurtado Anampa] 12:14:10
On the other cloud in double your Cg. Discussions how to make the cloud.