WEBVTT

1
00:00:02.669 --> 00:00:03.780
Matthew Bellis: Okay, I think you're good.

2
00:00:06.629 --> 00:00:08.160
Edgar Fernando Carrera Jarrin: Okay, very good.

3
00:00:09.780 --> 00:00:10.620
Edgar Fernando Carrera Jarrin: So.

4
00:00:11.730 --> 00:00:23.640
Edgar Fernando Carrera Jarrin: The the hands on session for the cloud computing remember last time we went I show you, even though I couldn't listen to you so sorry for that I couldn't take your questions.

5
00:00:25.200 --> 00:00:29.130
Edgar Fernando Carrera Jarrin: But I was trying to show you how to log into this.

6
00:00:30.810 --> 00:00:56.100
Edgar Fernando Carrera Jarrin: Google cloud platform and and it was really simple to start doing some tests we created our own cluster we we we just submitted some commands this keeps the DL command that allows us to to monitor whatever is happening in our cluster and.

7
00:00:57.540 --> 00:01:17.520
Edgar Fernando Carrera Jarrin: And that was the idea, and then I kind of hinted to you that, in order to be productive or really use seriously this is this those resources, then we had to start implementing or installing certain tools, the first tool that I mentioned, with our ago.

8
00:01:18.600 --> 00:01:26.460
Edgar Fernando Carrera Jarrin: We didn't have a chance to go through that and also we didn't have time to go through, for instance, creating a disc.

9
00:01:27.120 --> 00:01:36.630
Edgar Fernando Carrera Jarrin: to attach it to your cluster and then you can write to that desk that requires also to have certain servers going on.

10
00:01:36.960 --> 00:01:48.060
Edgar Fernando Carrera Jarrin: These nfl servers you know this network file system or whatever servers so you can communicate with the disk and all those kind of things.

11
00:01:48.750 --> 00:01:59.340
Edgar Fernando Carrera Jarrin: really take time and they are not super easy to learn, but it is possible, so you can do that this for yourself I partially.

12
00:01:59.910 --> 00:02:10.800
Edgar Fernando Carrera Jarrin: did some of that if you want to follow, but certainly in the workshop from from last year, you can find more information, some some things are probably outdated.

13
00:02:11.520 --> 00:02:19.980
Edgar Fernando Carrera Jarrin: But it's just things like you know the API version changes from year to year, or something like that, and they stopped being.

14
00:02:20.520 --> 00:02:35.280
Edgar Fernando Carrera Jarrin: The CMO files that we use this stuff working like otherwise they are good so i'm going to add here a link to the those other instructions in case you want to start learning about setting up your own cluster with all the toys.

15
00:02:36.720 --> 00:02:38.490
Edgar Fernando Carrera Jarrin: But we have done that for you.

16
00:02:39.510 --> 00:02:40.830
Edgar Fernando Carrera Jarrin: Already and.

17
00:02:42.360 --> 00:02:51.780
Edgar Fernando Carrera Jarrin: If you go to the session, the first session of the of the hands on tutorial which is called accessing your Internet is a cluster.

18
00:02:52.140 --> 00:03:05.670
Edgar Fernando Carrera Jarrin: You will find the first thing that you will find our instructions on how well I I recommend you you as for the other hands on material, I recommend that you read this.

19
00:03:06.150 --> 00:03:15.960
Edgar Fernando Carrera Jarrin: very carefully so start with that, and then the first thing would be to how to join this project, remember that.

20
00:03:16.440 --> 00:03:26.730
Edgar Fernando Carrera Jarrin: I was not working really in the project in the cms up a data project and last time I was working on my own area, but this time we have this.

21
00:03:27.420 --> 00:03:39.120
Edgar Fernando Carrera Jarrin: cms open data project within the Google cloud platform and in order to join this project, you have to follow those instructions that are set up in Google is lights.

22
00:03:39.540 --> 00:03:46.320
Edgar Fernando Carrera Jarrin: Please be careful not to change this, because everyone can change right now the information just skim over the.

23
00:03:47.190 --> 00:03:58.890
Edgar Fernando Carrera Jarrin: The slides and follow the instructions and then you, you should be good to go all right, I will give you a few minutes to do this and then.

24
00:03:59.400 --> 00:04:17.880
Edgar Fernando Carrera Jarrin: we'll go through this perhaps together, mostly it will be mostly you, but I like to pay attention and if gaddy and clemens can do the same, while you work on these instructions it'll be great okay.

25
00:04:27.330 --> 00:04:36.150
Edgar Fernando Carrera Jarrin: So it'll be good that you raise your hand when you're done with this step of just signing in so we know that you're ready.

26
00:04:37.440 --> 00:04:49.050
Edgar Fernando Carrera Jarrin: you're ready joined the project, and you can, of course, start working through the instructions and if something is not clear then just let us know.

27
00:04:50.460 --> 00:04:51.480
Edgar Fernando Carrera Jarrin: We can clarify.

28
00:06:11.400 --> 00:06:16.050
Matthew Bellis: grabbing quick question this might be for you in katia if we sent an alternate email.

29
00:06:18.810 --> 00:06:27.390
Matthew Bellis: Like I did this morning at this point here, where we click the log on button and we go to the CERN a single sign on page.

30
00:06:28.860 --> 00:06:31.410
Matthew Bellis: Do we enter in this alternate email address.

31
00:06:32.640 --> 00:06:39.930
Kati Lassila-Perini: You if you login with your gmail account think you should choose Google.

32
00:06:41.160 --> 00:06:42.930
Matthew Bellis: So I see what I see what you.

33
00:06:42.990 --> 00:06:44.430
Matthew Bellis: got it, thank you, thank you.

34
00:06:46.950 --> 00:06:48.240
Edgar Fernando Carrera Jarrin: yeah sounds reasonable.

35
00:07:13.980 --> 00:07:14.940
Matthew Bellis: yep that works, thank you.

36
00:07:34.830 --> 00:07:40.530
For me, none of the cloud computing pages are working is this occurring.

37
00:07:42.210 --> 00:07:47.460
With only me, or is there any problem with that.

38
00:07:50.100 --> 00:07:59.700
Edgar Fernando Carrera Jarrin: computing and the cloud computing links given on the main cms open data workshop none of them is working for me.

39
00:08:00.840 --> 00:08:01.950
Edgar Fernando Carrera Jarrin: Oh, you mean.

40
00:08:03.990 --> 00:08:04.350
Like.

41
00:08:06.240 --> 00:08:07.650
For example.

42
00:08:12.090 --> 00:08:13.380
For example, the.

43
00:08:14.490 --> 00:08:31.050
There is a page like data sets and accents So if you click a whole information emerges out like the sub topics under this article, but the cloud computing length is not giving any.

44
00:08:32.430 --> 00:08:37.230
Information it says that it is not able to find the page.

45
00:08:40.860 --> 00:08:59.310
Kati Lassila-Perini: Would you might be send a message through matter most with the credentials, you gave the so I can follow up with the with the people see if you send me got the got the last celebrity in the in the matter most I can continue matter most.

46
00:09:28.800 --> 00:09:30.900
Edgar Fernando Carrera Jarrin: don't forget to raise your hand if you.

47
00:09:31.290 --> 00:09:32.700
If you join the project.

48
00:10:03.480 --> 00:10:10.050
Dillon Fitzgerald: yeah i'm having trouble getting nice their credentials accepted try around two browsers now.

49
00:10:11.490 --> 00:10:14.250
Just says cloud bank logins are limited.

50
00:10:15.480 --> 00:10:17.370
And I had an accountability to be.

51
00:10:28.290 --> 00:10:30.600
Edgar Fernando Carrera Jarrin: And this is this is, this is the.

52
00:10:30.630 --> 00:10:32.310
Edgar Fernando Carrera Jarrin: credentials that you.

53
00:10:33.150 --> 00:10:41.010
Edgar Fernando Carrera Jarrin: submitted in this survey, we asked for the email that they needed for this, or I don't recall.

54
00:10:46.260 --> 00:10:52.350
Matthew Bellis: yeah I think that clements had asked for alternate emails if.

55
00:10:53.580 --> 00:10:59.460
Clemens Lange: If it's different from the indigo account yeah so if if.

56
00:10:59.940 --> 00:11:01.800
Clemens Lange: I mean principal your indigo account is.

57
00:11:02.040 --> 00:11:04.020
Clemens Lange: Linked to the sun single sign on.

58
00:11:04.980 --> 00:11:13.650
Clemens Lange: Unless you know deliberately chosen not to have that and then principle, the initial work, but I mean if if.

59
00:11:15.270 --> 00:11:20.850
Clemens Lange: If you're, for instance, not using single sign on single sign on or I don't have a son account and.

60
00:11:22.770 --> 00:11:30.450
Clemens Lange: You know, you might have to provide us with your email address, and also an alternate authentic vacation method Maybe you can find your Institute.

61
00:11:32.280 --> 00:11:33.840
Clemens Lange: mean the best thing will be to use.

62
00:11:34.920 --> 00:11:43.230
Clemens Lange: Google, if you have a Google account and if your son account doesn't work, but then you need to let us know what is the email address you're using.

63
00:11:46.080 --> 00:11:49.350
Dillon Fitzgerald: I see it might be the issues that I have.

64
00:11:50.490 --> 00:11:56.790
Multiple accounts that thing so maybe the wrong address provided to.

65
00:11:59.940 --> 00:12:09.240
Matthew Bellis: And that was my issue actually also that that my indigo account has a few different emails associated with it, so I provided.

66
00:12:10.350 --> 00:12:25.710
Matthew Bellis: Coffee this morning with my just regular standard gmail like personal gmail address, and then, when it came up for the in the process, just now, for the authentication I instead of choosing crn I just chose Google.

67
00:12:27.660 --> 00:12:43.050
Matthew Bellis: So to clements and contact, if this is an issue for people are they able to add their emails would you prefer them to do it through direct message and matter most or in zoom or is it a little bit late.

68
00:12:47.370 --> 00:12:56.160
Clemens Lange: Well, we can try to add, but I mean the thing is that you know the email addresses and everything, so I mean if let's say if you try for it, if you tried with sun.

69
00:12:56.970 --> 00:13:04.500
Clemens Lange: doesn't doesn't work, I mean one thing you can do is you go to the same directory and you check what is the actual email address registered there and then.

70
00:13:04.740 --> 00:13:07.830
Clemens Lange: Add to the one that you use when you're a new yorker account and then.

71
00:13:10.800 --> 00:13:14.580
Clemens Lange: yeah that's something we can try out and otherwise, you have to give us your.

72
00:13:16.290 --> 00:13:26.340
Clemens Lange: updated or corrected email address which, for instance, if you have a like gmail account or something, and we can also use that one but yeah it's not on our hands to register this so.

73
00:13:27.150 --> 00:13:27.840
Clemens Lange: We can.

74
00:13:27.930 --> 00:13:35.880
Clemens Lange: You know we'll try to forward the email to the person responsible and then hopefully you know they will be able to read quickly.

75
00:13:36.570 --> 00:13:37.380
Matthew Bellis: Okay, thanks.

76
00:14:37.200 --> 00:14:44.550
Edgar Fernando Carrera Jarrin: So just to be clear, they have to use their the the mail account with which they sign in to the indigo right.

77
00:14:44.610 --> 00:14:46.680
Edgar Fernando Carrera Jarrin: that's the base plan.

78
00:14:48.540 --> 00:14:50.580
Clemens Lange: yeah that's the one that we.

79
00:14:51.810 --> 00:14:52.440
Clemens Lange: Use now.

80
00:14:53.820 --> 00:14:54.210
Dillon Fitzgerald: Yes.

81
00:14:54.870 --> 00:14:56.490
Edgar Fernando Carrera Jarrin: it's it's not like.

82
00:14:57.690 --> 00:15:08.190
Edgar Fernando Carrera Jarrin: This is for addition it's not like I suppose it's not like you can just switch to your gmail you have to I mean you're subscribed and authorized.

83
00:15:08.250 --> 00:15:10.260
Edgar Fernando Carrera Jarrin: In this cloud bank.

84
00:15:11.220 --> 00:15:13.530
Edgar Fernando Carrera Jarrin: email for the indigo.

85
00:15:16.500 --> 00:15:17.430
Clemens Lange: gmail address.

86
00:15:19.290 --> 00:15:20.430
Clemens Lange: Trying to try to use.

87
00:15:23.430 --> 00:15:24.780
Clemens Lange: Google, instead of using sound.

88
00:15:26.490 --> 00:15:26.970
Clemens Lange: fix it.

89
00:15:32.940 --> 00:15:39.780
Clemens Lange: Basically, the rule of thumb is if your email addresses you know like@gmail.com try to use the Google and.

90
00:15:40.980 --> 00:15:44.850
Clemens Lange: Education provider, not the second one, if your email address ends and send.

91
00:15:48.030 --> 00:15:49.170
Clemens Lange: us the sun sign on.

92
00:16:08.130 --> 00:16:14.880
Dillon Fitzgerald: Yes, I think the firm that I was able to authenticate via my university email because that's the only one that is linked to my.

93
00:16:21.060 --> 00:16:21.510
Edgar Fernando Carrera Jarrin: Great.

94
00:18:11.700 --> 00:18:24.210
Edgar Fernando Carrera Jarrin: yeah don't forget to raise your hand it will be great to know how many people are having problems, so far, we are more than 30 people and in just a few hands are up.

95
00:18:50.520 --> 00:18:57.540
Dillon Fitzgerald: I don't have any thing that account, I mean, I was able to follow the prompts and everything with my mission email but.

96
00:18:59.010 --> 00:19:04.560
Dillon Fitzgerald: I don't get the same landing page after clicking on the building and exits.

97
00:19:32.700 --> 00:19:41.430
Clemens Lange: So, so if you don't if that billing account access pages empty for you then there's a problem with your email address association.

98
00:19:43.140 --> 00:20:02.580
Clemens Lange: So um while while we can try to get in touch with certain it and get that email address updated what we would need to know your email address you have to put it into the chat I mean we're we're basically copying the email addresses that are ending up in either matter most or.

99
00:20:04.500 --> 00:20:05.040
Clemens Lange: zoom.

100
00:20:06.330 --> 00:20:17.880
Clemens Lange: And so that we can and then forward them, so you have to put your email address in either of these chat boxes, we can can take care of it but yeah we basically we hope it's going to walk.

101
00:20:21.660 --> 00:20:25.290
Dillon Fitzgerald: US though for resisting in the email addresses.

102
00:20:27.360 --> 00:20:31.260
Dillon Fitzgerald: associated with their account for the billing landing page to work properly.

103
00:20:34.320 --> 00:20:36.060
Dillon Fitzgerald: Because I do a register fine.

104
00:20:40.530 --> 00:20:45.270
Dillon Fitzgerald: I just imagine that my initial email is not necessarily connected devices.

105
00:20:51.270 --> 00:20:57.090
Clemens Lange: yeah I mean, I have to check what what is in the export from indy code and, in this case.

106
00:20:58.290 --> 00:20:59.820
Clemens Lange: To see what what email address you.

107
00:21:00.840 --> 00:21:01.620
Clemens Lange: can have a look.

108
00:21:03.540 --> 00:21:08.520
Dillon Fitzgerald: Oh no sorry I already have that's why I was able to register with a human female I just.

109
00:21:09.930 --> 00:21:15.330
Dillon Fitzgerald: I don't know how to proceed now because there's nothing in the building page I could give you my surname email if that would help.

110
00:21:37.260 --> 00:21:39.780
Franz Glessgen: workflow DEMO file that's okay.

111
00:21:42.690 --> 00:21:43.350
Edgar Fernando Carrera Jarrin: yeah go ahead.

112
00:21:44.160 --> 00:21:51.390
Franz Glessgen: Okay, so this in the step of the right file is that there is some files, the text.

113
00:21:53.070 --> 00:21:55.650
Franz Glessgen: In the Mount roll directory.

114
00:22:05.250 --> 00:22:09.780
Franz Glessgen: it's when once once this workflow we are go unified has been downloaded.

115
00:22:10.650 --> 00:22:14.310
Franz Glessgen: And yes, with the different steps coded in jamo.

116
00:22:16.380 --> 00:22:20.760
Franz Glessgen: Yes, lately, oh no, this is the test right Yes, this is the test.

117
00:22:22.680 --> 00:22:23.010
Franz Glessgen: and

118
00:22:25.950 --> 00:22:42.930
Franz Glessgen: Right and I just want to ask there is this to to open the to get the file locations there's this this the reference that is used in this is that in this file, which is called files underscore some number that text and where does this file live.

119
00:22:44.190 --> 00:22:45.960
Franz Glessgen: Because we didn't write it right.

120
00:22:50.730 --> 00:22:51.450
Edgar Fernando Carrera Jarrin: Sorry.

121
00:22:53.940 --> 00:22:56.940
Franz Glessgen: it's in the Spice spice things up.

122
00:22:57.390 --> 00:22:59.070
Edgar Fernando Carrera Jarrin: Or is it ready yeah okay.

123
00:23:01.080 --> 00:23:01.620
Edgar Fernando Carrera Jarrin: Okay.

124
00:23:01.680 --> 00:23:06.120
Franz Glessgen: yeah right exactly in this workshop the ARGO the GMO fight this guy.

125
00:23:07.380 --> 00:23:11.700
Edgar Fernando Carrera Jarrin: This one right here, yes you're saying that this is not there.

126
00:23:12.240 --> 00:23:14.130
Franz Glessgen: No, this is there this is fine now i'm looking at the.

127
00:23:14.130 --> 00:23:14.520
Franz Glessgen: finance.

128
00:23:15.480 --> 00:23:17.940
Franz Glessgen: And I was just having a question about one of the steps.

129
00:23:19.470 --> 00:23:29.550
Edgar Fernando Carrera Jarrin: Oh, I see I see you when you when you're almost done with is always there was thinking that it was a narrative stage okay so yeah what what's.

130
00:23:29.760 --> 00:23:33.540
Franz Glessgen: Okay, no, you can check in the night in the right size that template.

131
00:23:35.040 --> 00:23:37.740
Edgar Fernando Carrera Jarrin: Yes, just give me a second I can.

132
00:23:37.830 --> 00:23:39.180
Franz Glessgen: Sure sure sure no problem is.

133
00:23:40.680 --> 00:23:44.100
Edgar Fernando Carrera Jarrin: Better version I suppose.

134
00:23:45.450 --> 00:23:47.730
Edgar Fernando Carrera Jarrin: color is this always better.

135
00:23:51.690 --> 00:23:52.740
Franz Glessgen: Yes, this is.

136
00:23:53.250 --> 00:23:55.590
Edgar Fernando Carrera Jarrin: This question is probably forgotten more than for me.

137
00:23:55.830 --> 00:23:58.560
Franz Glessgen: Okay, go ahead okay yeah so in the.

138
00:23:59.730 --> 00:24:01.230
Franz Glessgen: Right file step template.

139
00:24:03.450 --> 00:24:04.470
Edgar Fernando Carrera Jarrin: Yes.

140
00:24:04.980 --> 00:24:05.970
Franz Glessgen: With the commands that.

141
00:24:06.360 --> 00:24:12.000
Franz Glessgen: is exactly yes, yes that's exactly so you read you read these files from this with the extra TD.

142
00:24:12.210 --> 00:24:18.180
Franz Glessgen: Extra the command and then you read them using a list of names, you can scroll to the right.

143
00:24:19.980 --> 00:24:20.010
Franz Glessgen: Please.

144
00:24:21.450 --> 00:24:29.040
Franz Glessgen: Yes, exactly yeah right, so this files on the score blah blah text so we didn't write this file, so this you prepared for us or.

145
00:24:30.210 --> 00:24:38.340
Kati Lassila-Perini: The this the scatter underscore fast, that is written here which file you referring to limit.

146
00:24:39.000 --> 00:24:40.290
Edgar Fernando Carrera Jarrin: So for this one.

147
00:24:40.800 --> 00:24:42.810
Franz Glessgen: yeah this right, right up there.

148
00:24:43.470 --> 00:24:50.070
Franz Glessgen: Just 190 yes exactly this guy is this is used to read to get the file locations for.

149
00:24:50.100 --> 00:24:51.540
Franz Glessgen: These are we going to use.

150
00:24:52.200 --> 00:24:55.200
Kati Lassila-Perini: This is taken from the.

151
00:24:56.550 --> 00:25:14.790
Kati Lassila-Perini: From this with the CERN open data client container that's a service that gives you all files that belong to that to that this get file location, if you encouraged to go to the start of the start of the line line line.

152
00:25:15.330 --> 00:25:16.950
Kati Lassila-Perini: Thank you, yes, so good.

153
00:25:17.550 --> 00:25:29.850
Kati Lassila-Perini: yeah, so we are in the sun open data clients container and we submit a command get five locations, with the record ID which is corresponding to the other file of interest, where we.

154
00:25:29.910 --> 00:25:39.030
Kati Lassila-Perini: Then be diffused that's the protocol that were given to us as X rudy so we have done in in a written in a format that just can block them in directly to the false.

155
00:25:39.390 --> 00:25:40.290
Franz Glessgen: Okay, how would we.

156
00:25:40.560 --> 00:25:44.910
Franz Glessgen: How would we if we want if we know which file, we want to access, how would we get this record ID from it.

157
00:25:45.720 --> 00:25:48.720
Kati Lassila-Perini: You get it from these certain open data portal.

158
00:25:48.960 --> 00:25:49.290
Franz Glessgen: Okay.

159
00:25:49.410 --> 00:25:51.270
Edgar Fernando Carrera Jarrin: Great so let me, let me show you.

160
00:25:53.220 --> 00:25:54.780
Edgar Fernando Carrera Jarrin: it's just you have to.

161
00:25:55.950 --> 00:26:07.560
Edgar Fernando Carrera Jarrin: know this is a good question you go to cms, for instance, and you go to collision right, and you have a data set let's say, one of the ones that we were dealing with.

162
00:26:08.820 --> 00:26:14.940
Edgar Fernando Carrera Jarrin: It doesn't really matter but say this double electron blah blah, and then you get a number here see.

163
00:26:15.750 --> 00:26:18.780
Franz Glessgen: Okay, this one, this one okay great okay thanks.

164
00:26:18.840 --> 00:26:20.850
Franz Glessgen: yeah yeah great.

165
00:26:30.840 --> 00:26:32.400
Matthew Bellis: conti this may be an issue that.

166
00:26:33.420 --> 00:26:45.720
Matthew Bellis: That Sarah and I both had some issues, because we have to give you our email addresses this morning, but when we look at the list of the name clusters, where it's like cluster and then somebody's name, neither of us sees one.

167
00:26:46.860 --> 00:26:52.350
Matthew Bellis: With anything that looks like it might have been our names or our last names are some collaboration.

168
00:26:54.630 --> 00:26:57.930
Kati Lassila-Perini: I have to check with the with the person who's responsible.

169
00:26:58.230 --> 00:26:58.500
Matthew Bellis: Okay.

170
00:26:58.860 --> 00:27:00.390
Kati Lassila-Perini: So I can't tell you the sunlight.

171
00:27:00.900 --> 00:27:03.540
Matthew Bellis: No worries Okay, maybe, maybe it'll pop up in a little while.

172
00:29:02.490 --> 00:29:11.490
Kati Lassila-Perini: Just a follow up for the people who are who gave the gmail actresses that weren't working, you will soon get an email.

173
00:29:12.690 --> 00:29:20.250
Kati Lassila-Perini: And then you will be able to connect through the Google cloud console directly, but we will follow up with that.

174
00:29:26.640 --> 00:29:27.180
Matthew Bellis: Thank you.

175
00:31:24.840 --> 00:31:25.860
Edgar Fernando Carrera Jarrin: We should go for a break.

176
00:31:33.000 --> 00:31:34.380
Edgar Fernando Carrera Jarrin: Is that's right in.

177
00:31:35.940 --> 00:31:37.050
Edgar Fernando Carrera Jarrin: Like five minutes right.

178
00:32:02.280 --> 00:32:04.350
Franz Glessgen: So I have a question just for the association.

179
00:32:06.900 --> 00:32:07.890
Edgar Fernando Carrera Jarrin: yeah so.

180
00:32:09.060 --> 00:32:18.360
Franz Glessgen: Okay, so now if we would would like to run this workflow with different nodes with it suffice to just to just change our configuration in the.

181
00:32:18.840 --> 00:32:28.770
Franz Glessgen: tcp and then it would take care of it automatically or can we assign different jobs to different nodes or do we have to do the paralleling Personalizing ourselves.

182
00:32:36.210 --> 00:32:51.810
Kati Lassila-Perini: This particular workflow which is shown on the screen, it takes care of the it's such a such a such an extent takes care of the paralyzing and depending on the amount of notes that you have, but you may.

183
00:32:52.920 --> 00:32:54.120
Franz Glessgen: Can you tell me where exactly.

184
00:32:54.810 --> 00:32:57.330
Kati Lassila-Perini: It is the scatter plot, so it takes.

185
00:32:59.010 --> 00:32:59.640
Franz Glessgen: You.

186
00:33:00.810 --> 00:33:01.320
Franz Glessgen: Okay, see.

187
00:33:01.920 --> 00:33:05.070
Kati Lassila-Perini: It creates generates an iterator in.

188
00:33:05.100 --> 00:33:05.400
Franz Glessgen: yeah.

189
00:33:05.490 --> 00:33:08.970
Kati Lassila-Perini: In the Python part, and these are these are than the steps that it will take.

190
00:33:09.780 --> 00:33:13.380
Franz Glessgen: Okay, and this is this part is paralyzed Okay, yes, great thanks.

191
00:33:15.030 --> 00:33:19.410
Edgar Fernando Carrera Jarrin: So scotty I actually have a question myself so.

192
00:33:22.260 --> 00:33:26.070
Edgar Fernando Carrera Jarrin: But let's say that you choose here that's a.

193
00:33:27.570 --> 00:33:31.050
Edgar Fernando Carrera Jarrin: If I understand this correctly, and this is.

194
00:33:32.070 --> 00:33:34.530
Edgar Fernando Carrera Jarrin: Your starting from the whole data set.

195
00:33:35.670 --> 00:33:42.180
Edgar Fernando Carrera Jarrin: From the whole list of files you're starting from the third one, and you just give four files to be process.

196
00:33:43.740 --> 00:33:52.440
Edgar Fernando Carrera Jarrin: Right, and so it will go from the third one to the seventh one, and then it will go through those files and that's it right, but.

197
00:33:52.980 --> 00:34:04.680
Edgar Fernando Carrera Jarrin: Yes, or how does it work and, if you want to go over all the files you put here one and then what would you put here the just a number, the final number, according to the open data portal.

198
00:34:05.700 --> 00:34:06.330
Kati Lassila-Perini: Yes.

199
00:34:06.690 --> 00:34:08.790
Edgar Fernando Carrera Jarrin: And then we'll and then we'll work.

200
00:34:09.600 --> 00:34:22.020
Kati Lassila-Perini: We probably need to check that the the the the Google, which works properly because at the moment, we are doing in a little bit of the example way just taking.

201
00:34:22.350 --> 00:34:26.730
Edgar Fernando Carrera Jarrin: Right right, I mean it just is the extreme, but what i'm thinking is that.

202
00:34:28.800 --> 00:34:30.780
Edgar Fernando Carrera Jarrin: It won't work because.

203
00:34:32.700 --> 00:34:36.750
Edgar Fernando Carrera Jarrin: If this cluster is created, so the cluster is created.

204
00:34:39.090 --> 00:34:50.130
Edgar Fernando Carrera Jarrin: I suppose is created with the scalable option that we used yesterday, but still, you have ranges now like you can grow infinitely.

205
00:34:50.670 --> 00:35:11.220
Edgar Fernando Carrera Jarrin: And if the scalability is up to four machines, then you only have four machines and so much memory and in each cms w job takes let's say two gigabytes of memory, on average, and without thinking of the overhead and then.

206
00:35:13.050 --> 00:35:19.950
Edgar Fernando Carrera Jarrin: If you put that many parts, maybe the machines will will say show this stuff working.

207
00:35:20.250 --> 00:35:23.820
Kati Lassila-Perini: know the limits limits will take care of that because.

208
00:35:24.870 --> 00:35:40.230
Kati Lassila-Perini: For example, this in the settings that we have, which may not be optimal it runs only one job before if we have four nodes it runs for parallel jobs and then it when those are done it takes the next four.

209
00:35:41.190 --> 00:35:42.750
Edgar Fernando Carrera Jarrin: Okay, and so.

210
00:35:46.260 --> 00:35:47.820
Edgar Fernando Carrera Jarrin: So where's that defined.

211
00:35:47.850 --> 00:35:49.350
Edgar Fernando Carrera Jarrin: I fail to notice.

212
00:35:50.310 --> 00:35:53.100
Kati Lassila-Perini: No that's the company for us.

213
00:35:53.460 --> 00:35:55.050
Edgar Fernando Carrera Jarrin: Oh OK set up.

214
00:35:55.080 --> 00:35:55.950
Kati Lassila-Perini: In the cluster.

215
00:35:56.160 --> 00:35:58.530
Edgar Fernando Carrera Jarrin: I see, so you are defining.

216
00:36:00.150 --> 00:36:11.010
Edgar Fernando Carrera Jarrin: Right so in the setup of the cluster you can you can tell, they each node can handle a maximum of this number of parts Is that correct.

217
00:36:11.730 --> 00:36:17.490
Kati Lassila-Perini: Well, we actually handling in the in the in the resource requests on the steps.

218
00:36:21.270 --> 00:36:22.260
Kati Lassila-Perini: Each tips has.

219
00:36:23.700 --> 00:36:31.440
Kati Lassila-Perini: has certain requests, so if if they say it's a but if the no destiny cannot fit more than one.

220
00:36:32.040 --> 00:36:33.270
Kati Lassila-Perini: More than one this was.

221
00:36:33.750 --> 00:36:36.570
Kati Lassila-Perini: The resources, the limits and the requests in particular.

222
00:36:37.710 --> 00:36:40.380
Edgar Fernando Carrera Jarrin: Right, so the limited limited to.

223
00:36:40.380 --> 00:36:45.690
Kati Lassila-Perini: That it can't it can't use more than that, and the requests, is what it wants.

224
00:36:47.040 --> 00:36:47.430
Edgar Fernando Carrera Jarrin: Okay.

225
00:36:48.630 --> 00:36:50.160
I learned something new myself.

226
00:36:52.110 --> 00:36:53.430
Edgar Fernando Carrera Jarrin: I hadn't dead attention.

227
00:36:53.430 --> 00:36:54.180
To this part.

228
00:36:57.210 --> 00:37:02.790
Franz Glessgen: So I have another quick question about the so to get this ARGO command line.

229
00:37:04.350 --> 00:37:14.700
Franz Glessgen: We need to download the script repository but now I did disconnect and reconnected and it's disappeared, so it is normal is expected.

230
00:37:16.530 --> 00:37:17.070
that's still.

231
00:37:18.180 --> 00:37:22.110
Edgar Fernando Carrera Jarrin: To me to do you mean you're not you will have to reinstall it right.

232
00:37:22.290 --> 00:37:23.550
Franz Glessgen: Yes, exactly that's what I mean.

233
00:37:24.090 --> 00:37:25.680
Kati Lassila-Perini: You usually have the.

234
00:37:27.090 --> 00:37:32.010
Kati Lassila-Perini: The your area is there, but it's the if you move it to the to the user been.

235
00:37:33.030 --> 00:37:34.170
Kati Lassila-Perini: If you move the executable.

236
00:37:35.340 --> 00:37:38.520
Franz Glessgen: Then oh yeah just need to be executed from my Okay, yes yeah.

237
00:37:39.660 --> 00:37:50.340
Kati Lassila-Perini: All you need to instead of moving it, they just copy and paste because your own area will will be there, but you can also use the ARGO graphical user interface, which is, which is.

238
00:37:53.340 --> 00:37:53.610
Thanks.

239
00:38:16.050 --> 00:38:17.610
Edgar Fernando Carrera Jarrin: So with this limits.

240
00:38:18.630 --> 00:38:24.600
Edgar Fernando Carrera Jarrin: This parallelism that we were using doesn't make sense anymore right I.

241
00:38:24.690 --> 00:38:37.890
Kati Lassila-Perini: mean it's will does make sense, because it still runs the jobs in in parallel in the notes in the margin notes well, you can you can optimize the you can optimize limits, you can, if the job.

242
00:38:39.720 --> 00:38:52.590
Kati Lassila-Perini: requires less you can you can these, these are not optimized necessarily so, you if you if it's nice less you can you can make them load, so it can soak more more more jobs can run one node.

243
00:38:55.950 --> 00:38:56.340
Kati Lassila-Perini: issue.

244
00:39:30.960 --> 00:39:35.130
Edgar Fernando Carrera Jarrin: So matter we stop the recording right now and and.

245
00:39:36.720 --> 00:39:38.160
Edgar Fernando Carrera Jarrin: And maybe go for a.

246
00:39:39.330 --> 00:39:41.790
Edgar Fernando Carrera Jarrin: While, at least officially go for the break.

247
00:39:42.660 --> 00:39:43.590
Matthew Bellis: sounds good to me.

248
00:39:44.250 --> 00:39:45.930
Edgar Fernando Carrera Jarrin: restart 10 minutes after.

249
00:39:54.780 --> 00:39:54.990
Disha Bhatia: This.

250
00:39:58.800 --> 00:39:59.250
Disha Bhatia: episode.

251
00:40:02.490 --> 00:40:02.970
Disha Bhatia: So.

252
00:40:04.770 --> 00:40:05.190
Disha Bhatia: This.

253
00:40:06.570 --> 00:40:08.370
Disha Bhatia: This can be good.

254
00:40:18.210 --> 00:40:22.080
Edgar Fernando Carrera Jarrin: I really had a problem understanding.

255
00:40:22.770 --> 00:40:24.810
Edgar Fernando Carrera Jarrin: I don't know if someone else picked up the question.

256
00:40:25.890 --> 00:40:31.920
Matthew Bellis: I wonder if I just because of the connection, I wonder if it might not be better if this question could be asked.

257
00:40:31.950 --> 00:40:36.390
Matthew Bellis: In the matter most channel, it might be easier for us to respond and.

258
00:40:36.390 --> 00:40:38.280
Matthew Bellis: be able to pick up all the question clearly.

259
00:40:38.730 --> 00:40:39.240
Yes.

260
00:40:41.460 --> 00:40:42.840
Matthew Bellis: Thank you very much thank you.

261
00:40:48.540 --> 00:40:52.290
Edgar Fernando Carrera Jarrin: yeah sorry the show you kind of are saturating.

262
00:40:55.320 --> 00:40:56.100
Edgar Fernando Carrera Jarrin: it's a bit.

263
00:40:56.160 --> 00:40:58.950
Edgar Fernando Carrera Jarrin: difficult to hear yeah but most should work.

264
00:40:59.550 --> 00:41:00.150
Disha Bhatia: Okay okay.

265
00:41:03.060 --> 00:41:03.870
Matthew Bellis: Thank you.

266
00:41:06.780 --> 00:41:16.080
Matthew Bellis: So i'm going to stop the video and we'll take a break after did you say 10 minutes or 15 minutes I don't have the schedule right in front of me.