Matthew Bellis: Okay, I think you're good.
Edgar Fernando Carrera Jarrin: Okay, very good.
Edgar Fernando Carrera Jarrin: So.
Edgar Fernando Carrera Jarrin: The the hands on session for the cloud computing remember last time we went I show you, even though I couldn't listen to you so sorry for that I couldn't take your questions.
Edgar Fernando Carrera Jarrin: But I was trying to show you how to log into this.
Edgar Fernando Carrera Jarrin: Google cloud platform and and it was really simple to start doing some tests we created our own cluster we we we just submitted some commands this keeps the DL command that allows us to to monitor whatever is happening in our cluster and.
Edgar Fernando Carrera Jarrin: And that was the idea, and then I kind of hinted to you that, in order to be productive or really use seriously this is this those resources, then we had to start implementing or installing certain tools, the first tool that I mentioned, with our ago.
Edgar Fernando Carrera Jarrin: We didn't have a chance to go through that and also we didn't have time to go through, for instance, creating a disc.
Edgar Fernando Carrera Jarrin: to attach it to your cluster and then you can write to that desk that requires also to have certain servers going on.
Edgar Fernando Carrera Jarrin: These nfl servers you know this network file system or whatever servers so you can communicate with the disk and all those kind of things.
Edgar Fernando Carrera Jarrin: really take time and they are not super easy to learn, but it is possible, so you can do that this for yourself I partially.
Edgar Fernando Carrera Jarrin: did some of that if you want to follow, but certainly in the workshop from from last year, you can find more information, some some things are probably outdated.
Edgar Fernando Carrera Jarrin: But it's just things like you know the API version changes from year to year, or something like that, and they stopped being.
Edgar Fernando Carrera Jarrin: The CMO files that we use this stuff working like otherwise they are good so i'm going to add here a link to the those other instructions in case you want to start learning about setting up your own cluster with all the toys.
Edgar Fernando Carrera Jarrin: But we have done that for you.
Edgar Fernando Carrera Jarrin: Already and.
Edgar Fernando Carrera Jarrin: If you go to the session, the first session of the of the hands on tutorial which is called accessing your Internet is a cluster.
Edgar Fernando Carrera Jarrin: You will find the first thing that you will find our instructions on how well I I recommend you you as for the other hands on material, I recommend that you read this.
Edgar Fernando Carrera Jarrin: very carefully so start with that, and then the first thing would be to how to join this project, remember that.
Edgar Fernando Carrera Jarrin: I was not working really in the project in the cms up a data project and last time I was working on my own area, but this time we have this.
Edgar Fernando Carrera Jarrin: cms open data project within the Google cloud platform and in order to join this project, you have to follow those instructions that are set up in Google is lights.
Edgar Fernando Carrera Jarrin: Please be careful not to change this, because everyone can change right now the information just skim over the.
Edgar Fernando Carrera Jarrin: The slides and follow the instructions and then you, you should be good to go all right, I will give you a few minutes to do this and then.
Edgar Fernando Carrera Jarrin: we'll go through this perhaps together, mostly it will be mostly you, but I like to pay attention and if gaddy and clemens can do the same, while you work on these instructions it'll be great okay.
Edgar Fernando Carrera Jarrin: So it'll be good that you raise your hand when you're done with this step of just signing in so we know that you're ready.
Edgar Fernando Carrera Jarrin: you're ready joined the project, and you can, of course, start working through the instructions and if something is not clear then just let us know.
Edgar Fernando Carrera Jarrin: We can clarify.
Matthew Bellis: grabbing quick question this might be for you in katia if we sent an alternate email.
Matthew Bellis: Like I did this morning at this point here, where we click the log on button and we go to the CERN a single sign on page.
Matthew Bellis: Do we enter in this alternate email address.
Kati Lassila-Perini: You if you login with your gmail account think you should choose Google.
Matthew Bellis: So I see what I see what you.
Matthew Bellis: got it, thank you, thank you.
Edgar Fernando Carrera Jarrin: yeah sounds reasonable.
Matthew Bellis: yep that works, thank you.
For me, none of the cloud computing pages are working is this occurring.
With only me, or is there any problem with that.
Edgar Fernando Carrera Jarrin: computing and the cloud computing links given on the main cms open data workshop none of them is working for me.
Edgar Fernando Carrera Jarrin: Oh, you mean.
Like.
For example.
For example, the.
There is a page like data sets and accents So if you click a whole information emerges out like the sub topics under this article, but the cloud computing length is not giving any.
Information it says that it is not able to find the page.
Kati Lassila-Perini: Would you might be send a message through matter most with the credentials, you gave the so I can follow up with the with the people see if you send me got the got the last celebrity in the in the matter most I can continue matter most.
Edgar Fernando Carrera Jarrin: don't forget to raise your hand if you.
If you join the project.
Dillon Fitzgerald: yeah i'm having trouble getting nice their credentials accepted try around two browsers now.
Just says cloud bank logins are limited.
And I had an accountability to be.
Edgar Fernando Carrera Jarrin: And this is this is, this is the.
Edgar Fernando Carrera Jarrin: credentials that you.
Edgar Fernando Carrera Jarrin: submitted in this survey, we asked for the email that they needed for this, or I don't recall.
Matthew Bellis: yeah I think that clements had asked for alternate emails if.
Clemens Lange: If it's different from the indigo account yeah so if if.
Clemens Lange: I mean principal your indigo account is.
Clemens Lange: Linked to the sun single sign on.
Clemens Lange: Unless you know deliberately chosen not to have that and then principle, the initial work, but I mean if if.
Clemens Lange: If you're, for instance, not using single sign on single sign on or I don't have a son account and.
Clemens Lange: You know, you might have to provide us with your email address, and also an alternate authentic vacation method Maybe you can find your Institute.
Clemens Lange: mean the best thing will be to use.
Clemens Lange: Google, if you have a Google account and if your son account doesn't work, but then you need to let us know what is the email address you're using.
Dillon Fitzgerald: I see it might be the issues that I have.
Multiple accounts that thing so maybe the wrong address provided to.
Matthew Bellis: And that was my issue actually also that that my indigo account has a few different emails associated with it, so I provided.
Matthew Bellis: Coffee this morning with my just regular standard gmail like personal gmail address, and then, when it came up for the in the process, just now, for the authentication I instead of choosing crn I just chose Google.
Matthew Bellis: So to clements and contact, if this is an issue for people are they able to add their emails would you prefer them to do it through direct message and matter most or in zoom or is it a little bit late.
Clemens Lange: Well, we can try to add, but I mean the thing is that you know the email addresses and everything, so I mean if let's say if you try for it, if you tried with sun.
Clemens Lange: doesn't doesn't work, I mean one thing you can do is you go to the same directory and you check what is the actual email address registered there and then.
Clemens Lange: Add to the one that you use when you're a new yorker account and then.
Clemens Lange: yeah that's something we can try out and otherwise, you have to give us your.
Clemens Lange: updated or corrected email address which, for instance, if you have a like gmail account or something, and we can also use that one but yeah it's not on our hands to register this so.
Clemens Lange: We can.
Clemens Lange: You know we'll try to forward the email to the person responsible and then hopefully you know they will be able to read quickly.
Matthew Bellis: Okay, thanks.
Edgar Fernando Carrera Jarrin: So just to be clear, they have to use their the the mail account with which they sign in to the indigo right.
Edgar Fernando Carrera Jarrin: that's the base plan.
Clemens Lange: yeah that's the one that we.
Clemens Lange: Use now.
Dillon Fitzgerald: Yes.
Edgar Fernando Carrera Jarrin: it's it's not like.
Edgar Fernando Carrera Jarrin: This is for addition it's not like I suppose it's not like you can just switch to your gmail you have to I mean you're subscribed and authorized.
Edgar Fernando Carrera Jarrin: In this cloud bank.
Edgar Fernando Carrera Jarrin: email for the indigo.
Clemens Lange: gmail address.
Clemens Lange: Trying to try to use.
Clemens Lange: Google, instead of using sound.
Clemens Lange: fix it.
Clemens Lange: Basically, the rule of thumb is if your email addresses you know like@gmail.com try to use the Google and.
Clemens Lange: Education provider, not the second one, if your email address ends and send.
Clemens Lange: us the sun sign on.
Dillon Fitzgerald: Yes, I think the firm that I was able to authenticate via my university email because that's the only one that is linked to my.
Edgar Fernando Carrera Jarrin: Great.
Edgar Fernando Carrera Jarrin: yeah don't forget to raise your hand it will be great to know how many people are having problems, so far, we are more than 30 people and in just a few hands are up.
Dillon Fitzgerald: I don't have any thing that account, I mean, I was able to follow the prompts and everything with my mission email but.
Dillon Fitzgerald: I don't get the same landing page after clicking on the building and exits.
Clemens Lange: So, so if you don't if that billing account access pages empty for you then there's a problem with your email address association.
Clemens Lange: So um while while we can try to get in touch with certain it and get that email address updated what we would need to know your email address you have to put it into the chat I mean we're we're basically copying the email addresses that are ending up in either matter most or.
Clemens Lange: zoom.
Clemens Lange: And so that we can and then forward them, so you have to put your email address in either of these chat boxes, we can can take care of it but yeah we basically we hope it's going to walk.
Dillon Fitzgerald: US though for resisting in the email addresses.
Dillon Fitzgerald: associated with their account for the billing landing page to work properly.
Dillon Fitzgerald: Because I do a register fine.
Dillon Fitzgerald: I just imagine that my initial email is not necessarily connected devices.
Clemens Lange: yeah I mean, I have to check what what is in the export from indy code and, in this case.
Clemens Lange: To see what what email address you.
Clemens Lange: can have a look.
Dillon Fitzgerald: Oh no sorry I already have that's why I was able to register with a human female I just.
Dillon Fitzgerald: I don't know how to proceed now because there's nothing in the building page I could give you my surname email if that would help.
Franz Glessgen: workflow DEMO file that's okay.
Edgar Fernando Carrera Jarrin: yeah go ahead.
Franz Glessgen: Okay, so this in the step of the right file is that there is some files, the text.
Franz Glessgen: In the Mount roll directory.
Franz Glessgen: it's when once once this workflow we are go unified has been downloaded.
Franz Glessgen: And yes, with the different steps coded in jamo.
Franz Glessgen: Yes, lately, oh no, this is the test right Yes, this is the test.
Franz Glessgen: and
Franz Glessgen: Right and I just want to ask there is this to to open the to get the file locations there's this this the reference that is used in this is that in this file, which is called files underscore some number that text and where does this file live.
Franz Glessgen: Because we didn't write it right.
Edgar Fernando Carrera Jarrin: Sorry.
Franz Glessgen: it's in the Spice spice things up.
Edgar Fernando Carrera Jarrin: Or is it ready yeah okay.
Edgar Fernando Carrera Jarrin: Okay.
Franz Glessgen: yeah right exactly in this workshop the ARGO the GMO fight this guy.
Edgar Fernando Carrera Jarrin: This one right here, yes you're saying that this is not there.
Franz Glessgen: No, this is there this is fine now i'm looking at the.
Franz Glessgen: finance.
Franz Glessgen: And I was just having a question about one of the steps.
Edgar Fernando Carrera Jarrin: Oh, I see I see you when you when you're almost done with is always there was thinking that it was a narrative stage okay so yeah what what's.
Franz Glessgen: Okay, no, you can check in the night in the right size that template.
Edgar Fernando Carrera Jarrin: Yes, just give me a second I can.
Franz Glessgen: Sure sure sure no problem is.
Edgar Fernando Carrera Jarrin: Better version I suppose.
Edgar Fernando Carrera Jarrin: color is this always better.
Franz Glessgen: Yes, this is.
Edgar Fernando Carrera Jarrin: This question is probably forgotten more than for me.
Franz Glessgen: Okay, go ahead okay yeah so in the.
Franz Glessgen: Right file step template.
Edgar Fernando Carrera Jarrin: Yes.
Franz Glessgen: With the commands that.
Franz Glessgen: is exactly yes, yes that's exactly so you read you read these files from this with the extra TD.
Franz Glessgen: Extra the command and then you read them using a list of names, you can scroll to the right.
Franz Glessgen: Please.
Franz Glessgen: Yes, exactly yeah right, so this files on the score blah blah text so we didn't write this file, so this you prepared for us or.
Kati Lassila-Perini: The this the scatter underscore fast, that is written here which file you referring to limit.
Edgar Fernando Carrera Jarrin: So for this one.
Franz Glessgen: yeah this right, right up there.
Franz Glessgen: Just 190 yes exactly this guy is this is used to read to get the file locations for.
Franz Glessgen: These are we going to use.
Kati Lassila-Perini: This is taken from the.
Kati Lassila-Perini: From this with the CERN open data client container that's a service that gives you all files that belong to that to that this get file location, if you encouraged to go to the start of the start of the line line line.
Kati Lassila-Perini: Thank you, yes, so good.
Kati Lassila-Perini: yeah, so we are in the sun open data clients container and we submit a command get five locations, with the record ID which is corresponding to the other file of interest, where we.
Kati Lassila-Perini: Then be diffused that's the protocol that were given to us as X rudy so we have done in in a written in a format that just can block them in directly to the false.
Franz Glessgen: Okay, how would we.
Franz Glessgen: How would we if we want if we know which file, we want to access, how would we get this record ID from it.
Kati Lassila-Perini: You get it from these certain open data portal.
Franz Glessgen: Okay.
Edgar Fernando Carrera Jarrin: Great so let me, let me show you.
Edgar Fernando Carrera Jarrin: it's just you have to.
Edgar Fernando Carrera Jarrin: know this is a good question you go to cms, for instance, and you go to collision right, and you have a data set let's say, one of the ones that we were dealing with.
Edgar Fernando Carrera Jarrin: It doesn't really matter but say this double electron blah blah, and then you get a number here see.
Franz Glessgen: Okay, this one, this one okay great okay thanks.
Franz Glessgen: yeah yeah great.
Matthew Bellis: conti this may be an issue that.
Matthew Bellis: That Sarah and I both had some issues, because we have to give you our email addresses this morning, but when we look at the list of the name clusters, where it's like cluster and then somebody's name, neither of us sees one.
Matthew Bellis: With anything that looks like it might have been our names or our last names are some collaboration.
Kati Lassila-Perini: I have to check with the with the person who's responsible.
Matthew Bellis: Okay.
Kati Lassila-Perini: So I can't tell you the sunlight.
Matthew Bellis: No worries Okay, maybe, maybe it'll pop up in a little while.
Kati Lassila-Perini: Just a follow up for the people who are who gave the gmail actresses that weren't working, you will soon get an email.
Kati Lassila-Perini: And then you will be able to connect through the Google cloud console directly, but we will follow up with that.
Matthew Bellis: Thank you.
Edgar Fernando Carrera Jarrin: We should go for a break.
Edgar Fernando Carrera Jarrin: Is that's right in.
Edgar Fernando Carrera Jarrin: Like five minutes right.
Franz Glessgen: So I have a question just for the association.
Edgar Fernando Carrera Jarrin: yeah so.
Franz Glessgen: Okay, so now if we would would like to run this workflow with different nodes with it suffice to just to just change our configuration in the.
Franz Glessgen: tcp and then it would take care of it automatically or can we assign different jobs to different nodes or do we have to do the paralleling Personalizing ourselves.
Kati Lassila-Perini: This particular workflow which is shown on the screen, it takes care of the it's such a such a such an extent takes care of the paralyzing and depending on the amount of notes that you have, but you may.
Franz Glessgen: Can you tell me where exactly.
Kati Lassila-Perini: It is the scatter plot, so it takes.
Franz Glessgen: You.
Franz Glessgen: Okay, see.
Kati Lassila-Perini: It creates generates an iterator in.
Franz Glessgen: yeah.
Kati Lassila-Perini: In the Python part, and these are these are than the steps that it will take.
Franz Glessgen: Okay, and this is this part is paralyzed Okay, yes, great thanks.
Edgar Fernando Carrera Jarrin: So scotty I actually have a question myself so.
Edgar Fernando Carrera Jarrin: But let's say that you choose here that's a.
Edgar Fernando Carrera Jarrin: If I understand this correctly, and this is.
Edgar Fernando Carrera Jarrin: Your starting from the whole data set.
Edgar Fernando Carrera Jarrin: From the whole list of files you're starting from the third one, and you just give four files to be process.
Edgar Fernando Carrera Jarrin: Right, and so it will go from the third one to the seventh one, and then it will go through those files and that's it right, but.
Edgar Fernando Carrera Jarrin: Yes, or how does it work and, if you want to go over all the files you put here one and then what would you put here the just a number, the final number, according to the open data portal.
Kati Lassila-Perini: Yes.
Edgar Fernando Carrera Jarrin: And then we'll and then we'll work.
Kati Lassila-Perini: We probably need to check that the the the the Google, which works properly because at the moment, we are doing in a little bit of the example way just taking.
Edgar Fernando Carrera Jarrin: Right right, I mean it just is the extreme, but what i'm thinking is that.
Edgar Fernando Carrera Jarrin: It won't work because.
Edgar Fernando Carrera Jarrin: If this cluster is created, so the cluster is created.
Edgar Fernando Carrera Jarrin: I suppose is created with the scalable option that we used yesterday, but still, you have ranges now like you can grow infinitely.
Edgar Fernando Carrera Jarrin: And if the scalability is up to four machines, then you only have four machines and so much memory and in each cms w job takes let's say two gigabytes of memory, on average, and without thinking of the overhead and then.
Edgar Fernando Carrera Jarrin: If you put that many parts, maybe the machines will will say show this stuff working.
Kati Lassila-Perini: know the limits limits will take care of that because.
Kati Lassila-Perini: For example, this in the settings that we have, which may not be optimal it runs only one job before if we have four nodes it runs for parallel jobs and then it when those are done it takes the next four.
Edgar Fernando Carrera Jarrin: Okay, and so.
Edgar Fernando Carrera Jarrin: So where's that defined.
Edgar Fernando Carrera Jarrin: I fail to notice.
Kati Lassila-Perini: No that's the company for us.
Edgar Fernando Carrera Jarrin: Oh OK set up.
Kati Lassila-Perini: In the cluster.
Edgar Fernando Carrera Jarrin: I see, so you are defining.
Edgar Fernando Carrera Jarrin: Right so in the setup of the cluster you can you can tell, they each node can handle a maximum of this number of parts Is that correct.
Kati Lassila-Perini: Well, we actually handling in the in the in the resource requests on the steps.
Kati Lassila-Perini: Each tips has.
Kati Lassila-Perini: has certain requests, so if if they say it's a but if the no destiny cannot fit more than one.
Kati Lassila-Perini: More than one this was.
Kati Lassila-Perini: The resources, the limits and the requests in particular.
Edgar Fernando Carrera Jarrin: Right, so the limited limited to.
Kati Lassila-Perini: That it can't it can't use more than that, and the requests, is what it wants.
Edgar Fernando Carrera Jarrin: Okay.
I learned something new myself.
Edgar Fernando Carrera Jarrin: I hadn't dead attention.
To this part.
Franz Glessgen: So I have another quick question about the so to get this ARGO command line.
Franz Glessgen: We need to download the script repository but now I did disconnect and reconnected and it's disappeared, so it is normal is expected.
that's still.
Edgar Fernando Carrera Jarrin: To me to do you mean you're not you will have to reinstall it right.
Franz Glessgen: Yes, exactly that's what I mean.
Kati Lassila-Perini: You usually have the.
Kati Lassila-Perini: The your area is there, but it's the if you move it to the to the user been.
Kati Lassila-Perini: If you move the executable.
Franz Glessgen: Then oh yeah just need to be executed from my Okay, yes yeah.
Kati Lassila-Perini: All you need to instead of moving it, they just copy and paste because your own area will will be there, but you can also use the ARGO graphical user interface, which is, which is.
Thanks.
Edgar Fernando Carrera Jarrin: So with this limits.
Edgar Fernando Carrera Jarrin: This parallelism that we were using doesn't make sense anymore right I.
Kati Lassila-Perini: mean it's will does make sense, because it still runs the jobs in in parallel in the notes in the margin notes well, you can you can optimize the you can optimize limits, you can, if the job.
Kati Lassila-Perini: requires less you can you can these, these are not optimized necessarily so, you if you if it's nice less you can you can make them load, so it can soak more more more jobs can run one node.
Kati Lassila-Perini: issue.
Edgar Fernando Carrera Jarrin: So matter we stop the recording right now and and.
Edgar Fernando Carrera Jarrin: And maybe go for a.
Edgar Fernando Carrera Jarrin: While, at least officially go for the break.
Matthew Bellis: sounds good to me.
Edgar Fernando Carrera Jarrin: restart 10 minutes after.
Disha Bhatia: This.
Disha Bhatia: episode.
Disha Bhatia: So.
Disha Bhatia: This.
Disha Bhatia: This can be good.
Edgar Fernando Carrera Jarrin: I really had a problem understanding.
Edgar Fernando Carrera Jarrin: I don't know if someone else picked up the question.
Matthew Bellis: I wonder if I just because of the connection, I wonder if it might not be better if this question could be asked.
Matthew Bellis: In the matter most channel, it might be easier for us to respond and.
Matthew Bellis: be able to pick up all the question clearly.
Yes.
Matthew Bellis: Thank you very much thank you.
Edgar Fernando Carrera Jarrin: yeah sorry the show you kind of are saturating.
Edgar Fernando Carrera Jarrin: it's a bit.
Edgar Fernando Carrera Jarrin: difficult to hear yeah but most should work.
Disha Bhatia: Okay okay.
Matthew Bellis: Thank you.
Matthew Bellis: So i'm going to stop the video and we'll take a break after did you say 10 minutes or 15 minutes I don't have the schedule right in front of me.