WEBVTT 1 00:00:01.260 --> 00:00:02.550 Matthew Bellis: And we're all set. 2 00:00:07.200 --> 00:00:07.589 Edgar Fernando Carrera Jarrin: yeah. 3 00:00:08.940 --> 00:00:10.469 Edgar Fernando Carrera Jarrin: got the requirements are. 4 00:00:12.059 --> 00:00:16.949 Edgar Fernando Carrera Jarrin: doing the best they can to get everyone set up hopefully. 5 00:00:18.180 --> 00:00:24.120 Edgar Fernando Carrera Jarrin: But you know these problems always appear when you're trying to massively set up. 6 00:00:25.470 --> 00:00:44.580 Edgar Fernando Carrera Jarrin: Some resources like this in any case don't worry, because we have time, we know we knew this problem, sometimes happen, and you will have time, once you get your account and your cluster access to your cluster, you will have time to go through the exercises. 7 00:00:45.720 --> 00:00:59.130 Edgar Fernando Carrera Jarrin: Meanwhile, if you are already working and have questions you know ask them, this is the opportunity, because we have the most experts here and and maybe we can all learn to thing. 8 00:17:09.360 --> 00:17:11.610 Cecilia Uribe Estrada: hello, can you hear hear me. 9 00:17:17.250 --> 00:17:17.730 Luca Giommi: Yes. 10 00:17:17.880 --> 00:17:18.750 Edgar Fernando Carrera Jarrin: hello, yes. 11 00:17:19.050 --> 00:17:20.070 Cecilia Uribe Estrada: Okay excellent. 12 00:17:20.130 --> 00:17:25.410 Cecilia Uribe Estrada: I have a question, so if we want to kill, for example, a job, what should we go. 13 00:17:27.120 --> 00:17:37.530 Edgar Fernando Carrera Jarrin: So there are, I mean you can see what I usually do is I open a cheat sheet, I can give you the answer right away, but then you can see my screen right. 14 00:17:38.460 --> 00:17:39.090 Cecilia Uribe Estrada: that's right. 15 00:17:40.710 --> 00:17:44.220 Edgar Fernando Carrera Jarrin: So I just killed one right now, so you go ARGO delete. 16 00:17:44.340 --> 00:17:51.480 Edgar Fernando Carrera Jarrin: poet or the name of the other part, right the name of the song Okay, let me do this more. 17 00:17:53.070 --> 00:18:13.050 Edgar Fernando Carrera Jarrin: systematically so what I usually do I, a list first my jobs right, so this one is running I don't want to kill it, but if I wanted to kill it I can I can choose this name the name of the workflow and then I can issue this delete with the name and the name space. 18 00:18:16.230 --> 00:18:19.020 Cecilia Uribe Estrada: I see okay perfect. 19 00:18:19.920 --> 00:18:30.450 Edgar Fernando Carrera Jarrin: I usually I usually pull up some some cheat sheets, you know I googled ARGO commands cheat sheet whatever because I never remember them. 20 00:18:34.080 --> 00:18:34.980 Cecilia Uribe Estrada: Thank you perfect. 21 00:18:48.120 --> 00:19:07.920 Edgar Fernando Carrera Jarrin: there's also this very useful command, which is our go get latest, so it on the focuses on your latest workflow and it gives you like get an expansion of what's going on, so if you have many steps, then you get you can check how they're going. 22 00:19:10.590 --> 00:19:12.480 Edgar Fernando Carrera Jarrin: that's great I have here for. 23 00:19:12.900 --> 00:19:21.180 Edgar Fernando Carrera Jarrin: For for parts right for parts running because I get one file, but instead they are going, in parallel, which is really nice. 24 00:19:23.640 --> 00:19:28.320 Edgar Fernando Carrera Jarrin: And you can also use gypsy do of course always to get the plants. 25 00:19:29.910 --> 00:19:31.770 Edgar Fernando Carrera Jarrin: And the namespace. 26 00:19:32.880 --> 00:19:37.200 Edgar Fernando Carrera Jarrin: And you should also be able to see them doing their thing. 27 00:19:38.250 --> 00:19:39.090 Edgar Fernando Carrera Jarrin: If this command. 28 00:19:44.070 --> 00:19:44.550 Cecilia Uribe Estrada: Thank you. 29 00:19:46.890 --> 00:19:47.130 Edgar Fernando Carrera Jarrin: know. 30 00:24:13.680 --> 00:24:16.050 pegah shandiz: hello, can you hear me. 31 00:24:17.910 --> 00:24:18.990 Edgar Fernando Carrera Jarrin: Yes, we can. 32 00:24:21.240 --> 00:24:27.060 pegah shandiz: I have a question if i'm using vpn it has an effect on a cluster IP or. 33 00:24:30.090 --> 00:24:32.670 pegah shandiz: Something like this external IP. 34 00:24:35.250 --> 00:24:37.980 Edgar Fernando Carrera Jarrin: So, again I couldn't get that. 35 00:24:38.430 --> 00:24:40.680 pegah shandiz: And if i'm using vpn. 36 00:24:42.240 --> 00:24:49.890 pegah shandiz: I want to know, and it has effect on cluster IP an external IP or not. 37 00:24:53.310 --> 00:24:53.670 pegah shandiz: I. 38 00:24:53.700 --> 00:24:57.150 Edgar Fernando Carrera Jarrin: don't know what is maybe 18 understanding well. 39 00:24:57.360 --> 00:25:02.490 Clemens Lange: No, no, no, I mean this, this is nothing I mean the vpn is on your side of the public. 40 00:25:02.970 --> 00:25:07.050 Clemens Lange: is on the Google side so there's no relation between the two. 41 00:25:07.500 --> 00:25:22.170 Clemens Lange: So it doesn't make a difference yeah I mean the IP that you get is the one that is on the Google server on you know, on the whether whether the computer Center is but the vpn is only relevant from you or from from where you basically access to the Internet, but. 42 00:25:23.400 --> 00:25:25.050 Clemens Lange: This has nothing to do so, some problem. 43 00:25:26.460 --> 00:25:37.170 pegah shandiz: Okay, thank you because when I am typing the external IP on my browser it doesn't work it doesn't let me to know. 44 00:25:39.180 --> 00:25:44.940 Clemens Lange: You can put the IP in the chat or in matter most and then someone else can try to. 45 00:25:46.020 --> 00:25:50.640 Clemens Lange: Access it but in principle to be like world readable so from it doesn't matter where you are. 46 00:25:51.810 --> 00:25:52.530 pegah shandiz: And thank you. 47 00:25:53.220 --> 00:25:55.260 Kati Lassila-Perini: If you follow the link from the. 48 00:25:56.850 --> 00:26:07.110 Kati Lassila-Perini: From the service list on the on the Google cloud platform side you would need to type it in the http is. 49 00:26:08.550 --> 00:26:09.390 Kati Lassila-Perini: In front of it. 50 00:26:11.880 --> 00:26:14.100 Edgar Fernando Carrera Jarrin: right which one, are you referring to the. 51 00:26:15.780 --> 00:26:17.520 Edgar Fernando Carrera Jarrin: web server or the or the web. 52 00:26:20.010 --> 00:26:20.580 pegah shandiz: server. 53 00:26:23.340 --> 00:26:24.450 Kati Lassila-Perini: That should be that's. 54 00:26:25.080 --> 00:26:25.680 pegah shandiz: Correct I will. 55 00:26:25.980 --> 00:26:28.230 Edgar Fernando Carrera Jarrin: Be I wouldn't goes without a problem yeah. 56 00:26:29.970 --> 00:26:30.480 pegah shandiz: Thank you. 57 00:26:32.010 --> 00:26:37.500 Edgar Fernando Carrera Jarrin: Okay, can you post the IP so so we can try, I can test whether I can access it. 58 00:26:37.560 --> 00:26:38.190 pegah shandiz: Yes, yes. 59 00:26:56.400 --> 00:26:57.570 hi it's me. 60 00:27:02.490 --> 00:27:03.390 Can you hear me. 61 00:27:05.760 --> 00:27:10.920 Edgar Fernando Carrera Jarrin: yeah okay i'm following the instructions given on this page. 62 00:27:13.200 --> 00:27:26.820 hands on accepting your key it cluster so we're i'm here, where you narrow down your search depending on your Google and Google ID so I can see. 63 00:27:29.190 --> 00:27:34.530 My name over there, and the number of nodes etc so over here they. 64 00:27:35.550 --> 00:27:47.040 have appointed or a pointer on this page is indicating that you need to connect So when I tried to connect It gives me a message. 65 00:27:48.270 --> 00:27:54.630 You need permissions for this action required permissions container dot clusters. 66 00:28:12.840 --> 00:28:14.130 Edgar Fernando Carrera Jarrin: You don't see this page. 67 00:28:16.620 --> 00:28:19.320 Edgar Fernando Carrera Jarrin: Sorry you're breaking up I don't know if it's only for me. 68 00:28:22.230 --> 00:28:31.950 Clemens Lange: No, sorry I stop talking, but I think I mean if it's asking you to give access, I mean you just click authorized and it's Okay, if you all, can you not see the pages that guy was asking. 69 00:28:34.740 --> 00:28:38.070 Clemens Lange: Okay, I mean it's best if you put the error message into the chat. 70 00:28:40.440 --> 00:28:41.400 Clemens Lange: For us to understand. 71 00:28:41.820 --> 00:28:44.310 teeba: Okay, like when I tried to connect. 72 00:28:45.480 --> 00:28:50.130 window, a small window opens on it and it gives me message. 73 00:28:51.450 --> 00:28:51.750 Like. 74 00:28:55.290 --> 00:28:56.580 How it seems. 75 00:28:57.600 --> 00:29:02.790 To be behaving in a different way seems it is connected connect to the cluster. 76 00:29:15.150 --> 00:29:24.870 Yes, you need permission for this section, the required permissions container dot clusters. 77 00:29:27.120 --> 00:29:41.670 Clemens Lange: More so than I think nine more another page opens up Okay, but I mean if you say container clusters did it, I mean when when does this happen, I mean you're trying to delete a container. 78 00:29:49.770 --> 00:29:50.190 Clemens Lange: Can you say. 79 00:30:01.980 --> 00:30:05.100 Clemens Lange: Actually, what you're saying to to perform. 80 00:30:06.960 --> 00:30:19.020 Edgar Fernando Carrera Jarrin: Your you know what happens is, if you hover over your mouse to the grayed out options, then that message message comes out so T is your connect button grayed out. 81 00:30:21.720 --> 00:30:22.050 No. 82 00:30:24.240 --> 00:30:24.480 Edgar Fernando Carrera Jarrin: But. 83 00:30:25.860 --> 00:30:27.840 Edgar Fernando Carrera Jarrin: The message windows appear. 84 00:30:29.310 --> 00:30:30.690 Edgar Fernando Carrera Jarrin: Okay, no, no, no. 85 00:30:34.890 --> 00:30:38.070 Edgar Fernando Carrera Jarrin: No, because if you put your hand your little head over. 86 00:30:49.620 --> 00:30:55.320 Edgar Fernando Carrera Jarrin: But because I do see that message see here, but only if I try to click on delete. 87 00:30:56.430 --> 00:31:03.480 Edgar Fernando Carrera Jarrin: But if I move just a little bit up and put my hand over connect I can click that button that's a different one. 88 00:31:05.910 --> 00:31:07.860 So when you click connect what happens. 89 00:31:09.510 --> 00:31:14.130 Edgar Fernando Carrera Jarrin: When I click connect It gives me the command that you should run independently. 90 00:31:15.540 --> 00:31:37.740 Edgar Fernando Carrera Jarrin: Okay, so Okay, should I do so now, I can see it yes Okay, so if you get this window you just click run in cloud Shell, so it will copy the whole command on the cloud Shell and then you will have to just click enter on the Shell and that will connect you with the with the cluster. 91 00:31:40.200 --> 00:31:41.160 Public cloud. 92 00:31:43.320 --> 00:31:43.890 Edgar Fernando Carrera Jarrin: that's right. 93 00:33:36.180 --> 00:33:39.780 Luca Giommi: Sorry, my eyes keep your question, can you hear me. 94 00:33:42.270 --> 00:33:43.170 Edgar Fernando Carrera Jarrin: Yes, please go ahead. 95 00:33:43.710 --> 00:34:03.900 Luca Giommi: Yes, I tried to run the workflow esteemed si and successfully, but my PDF file or the plot is empty So if I just submitted as a run in the in the young file, as I don't allow people without changing anything so yes. 96 00:34:05.670 --> 00:34:06.360 Luca Giommi: I would think this. 97 00:34:06.570 --> 00:34:12.780 Edgar Fernando Carrera Jarrin: yeah it's it, it seems to be just a glitch on the on the. 98 00:34:14.670 --> 00:34:16.290 Edgar Fernando Carrera Jarrin: On the on the code in. 99 00:34:17.610 --> 00:34:18.840 Edgar Fernando Carrera Jarrin: The code that is running. 100 00:34:19.380 --> 00:34:20.040 Kati Lassila-Perini: check number. 101 00:34:20.670 --> 00:34:21.720 Edgar Fernando Carrera Jarrin: He added this. 102 00:34:23.250 --> 00:34:24.570 Kati Lassila-Perini: chick number of events. 103 00:34:26.760 --> 00:34:27.780 Luca Giommi: Okay, let me check. 104 00:34:34.860 --> 00:34:41.910 Edgar Fernando Carrera Jarrin: I don't know how its produced it is something that got the added just a few a few couple of hours ago, maybe. 105 00:34:43.260 --> 00:34:43.860 Edgar Fernando Carrera Jarrin: I don't know that's. 106 00:34:43.920 --> 00:34:50.490 Kati Lassila-Perini: that's quite OK, this is this is perfectly normal I said, if you just check the number of events, you will understand. 107 00:34:52.290 --> 00:34:54.840 Kati Lassila-Perini: If you see that they called you can see the default. 108 00:34:58.290 --> 00:34:58.890 Kati Lassila-Perini: workflow. 109 00:34:59.430 --> 00:35:01.710 Edgar Fernando Carrera Jarrin: Oh let's see it came with a trick you say. 110 00:35:04.080 --> 00:35:05.010 Kati Lassila-Perini: accidentally. 111 00:35:06.780 --> 00:35:07.200 Edgar Fernando Carrera Jarrin: Okay. 112 00:35:11.550 --> 00:35:15.930 Edgar Fernando Carrera Jarrin: This is very good, it just forces us to to see. 113 00:35:42.450 --> 00:35:47.250 Kati Lassila-Perini: justin top of the workflow file, they have few parameters which i'll get into. 114 00:35:48.600 --> 00:35:49.110 Luca Giommi: yeah. 115 00:37:23.490 --> 00:37:36.540 Cecilia Uribe Estrada: hi yes, we are sorry probably you're ready playing these, but it was doing something else, so in the part where, when you say that we got to the second open data client tool. 116 00:37:39.900 --> 00:37:40.560 Edgar Fernando Carrera Jarrin: Right. 117 00:37:43.770 --> 00:37:50.670 Cecilia Uribe Estrada: So for the for the least of the files that we want to process probably your recipe that sorry I missed that. 118 00:37:51.840 --> 00:37:53.970 Edgar Fernando Carrera Jarrin: yeah this this part right. 119 00:37:54.360 --> 00:37:55.260 Cecilia Uribe Estrada: yeah I think so. 120 00:37:58.560 --> 00:37:59.010 Edgar Fernando Carrera Jarrin: yeah. 121 00:38:00.780 --> 00:38:01.590 Edgar Fernando Carrera Jarrin: read about it. 122 00:38:02.790 --> 00:38:03.060 Edgar Fernando Carrera Jarrin: yeah. 123 00:38:03.090 --> 00:38:17.370 Cecilia Uribe Estrada: So they saw it will I got to the tool, obviously I have a very big list of files So what are the ones that we want to to get. 124 00:38:19.050 --> 00:38:20.370 Cecilia Uribe Estrada: Work in our example. 125 00:38:21.390 --> 00:38:22.800 Edgar Fernando Carrera Jarrin: Well yeah so. 126 00:38:22.860 --> 00:38:27.270 Edgar Fernando Carrera Jarrin: The previous question if you're referring to that was how do you get this. 127 00:38:28.320 --> 00:38:32.490 Edgar Fernando Carrera Jarrin: record ID that is mentioned here, which is passed as a parameter. 128 00:38:33.300 --> 00:38:33.840 Cecilia Uribe Estrada: that's right. 129 00:38:35.580 --> 00:38:36.540 Edgar Fernando Carrera Jarrin: Right so. 130 00:38:37.140 --> 00:38:38.130 Edgar Fernando Carrera Jarrin: Okay, so. 131 00:38:38.580 --> 00:38:48.750 Edgar Fernando Carrera Jarrin: And the the answer was that if you go to any of these data sets right in on the open data portal, the record ID is this number that you see up here. 132 00:38:49.140 --> 00:38:54.300 Cecilia Uribe Estrada: I see okay uh huh yeah I think he was playing down not ready sorry yeah. 133 00:38:55.440 --> 00:38:59.040 Edgar Fernando Carrera Jarrin: No, no, no, not a problem, I mean everyone is doing their own thing and. 134 00:38:59.490 --> 00:39:01.740 Edgar Fernando Carrera Jarrin: Question repeats itself okay. 135 00:39:04.020 --> 00:39:07.320 Edgar Fernando Carrera Jarrin: yeah was that your question, though, or was there something else. 136 00:39:08.130 --> 00:39:11.280 Cecilia Uribe Estrada: yeah so once you identify the number. 137 00:39:12.330 --> 00:39:22.380 Cecilia Uribe Estrada: You So then, what do you do you select the files that you need or how do, how do we see the list of the files that we want to process. 138 00:39:23.910 --> 00:39:36.330 Edgar Fernando Carrera Jarrin: Well, this line actually is getting all the the file location, meaning when I presume, it does, because I haven't tried that what I presume, it does is it will get you to this list. 139 00:39:39.000 --> 00:39:41.670 Edgar Fernando Carrera Jarrin: To this list for all the files in the data set. 140 00:39:43.260 --> 00:39:49.710 Edgar Fernando Carrera Jarrin: So this is what you're dumped in essentially to these files text. 141 00:39:51.810 --> 00:39:57.060 Edgar Fernando Carrera Jarrin: Okay, so essentially you're getting all the list of the files in that. 142 00:39:58.860 --> 00:39:59.610 Edgar Fernando Carrera Jarrin: Data set. 143 00:40:00.930 --> 00:40:08.550 Edgar Fernando Carrera Jarrin: And then, and then you don't run over all the files in the data set because you asked the configuration to. 144 00:40:09.630 --> 00:40:19.500 Edgar Fernando Carrera Jarrin: To just start in the third file that say and just process for by default, that you can change a suppose these numbers and. 145 00:40:21.090 --> 00:40:23.370 Edgar Fernando Carrera Jarrin: then start running in more files. 146 00:40:24.570 --> 00:40:26.160 over the whole dataset. 147 00:40:30.630 --> 00:40:31.530 Cecilia Uribe Estrada: Okay, thank you. 148 00:50:48.600 --> 00:50:48.900 Oh. 149 00:51:33.390 --> 00:51:33.600 Better. 150 01:23:27.330 --> 01:23:31.080 Edgar Fernando Carrera Jarrin: Okay officially it's time for the second break and. 151 01:23:32.640 --> 01:23:46.050 Edgar Fernando Carrera Jarrin: You can go for a coffee and maybe when we come back in 15 minutes we can ask around whether people have been successful, maybe open the microphones to. 152 01:23:46.890 --> 01:24:00.570 Edgar Fernando Carrera Jarrin: To hear their comments and feedback on this particular exercise and if people need more time we can certainly take time off the next session the closing session, what do you think. 153 01:24:03.690 --> 01:24:04.770 Matthew Bellis: that's a good idea. 154 01:24:08.850 --> 01:24:09.960 Edgar Fernando Carrera Jarrin: Alright, so. 155 01:24:11.280 --> 01:24:15.750 Edgar Fernando Carrera Jarrin: we're still around so stop asking questions if you if you have them. 156 01:24:21.120 --> 01:24:21.450 Matthew Bellis: file. 157 01:24:25.620 --> 01:24:27.930 Chokri Ben Romdhane: And then we'll start it started again after.