WEBVTT

1
00:01:43.860 --> 00:01:45.810
Kati Lassila-Perini: Just let me know when you want to meet to stop.

2
00:01:46.770 --> 00:01:56.310
Matthew Bellis: I think I think we're all good i'll keep an eye on the waiting room let people in, and I think I think you can go ahead and kick this workshop off.

3
00:01:57.840 --> 00:02:07.200
Kati Lassila-Perini: Okay, thank you matt so it's my pleasure to welcome you to the same as open data workshop, this is the second edition of this type and i'm got the last laugh at me and.

4
00:02:07.740 --> 00:02:25.800
Kati Lassila-Perini: i'm coordinating the cms data preservation and open access group which is in charge of providing you this open data and i'm doing this on behalf of my colleagues, so the same as open data team that involved in the preparation of this workshop, and you certainly seen.

5
00:02:26.820 --> 00:02:37.770
Kati Lassila-Perini: All right, all of us in the in the in the matter most forum, we have matt Ellis clemence lung a Julie hogan if CARA CARA.

6
00:02:39.060 --> 00:02:53.490
Kati Lassila-Perini: And we've been preparing the material for this workshop in the sea in the last month or or even more if you take the time from the last workshop what she had in in October.

7
00:02:57.420 --> 00:03:02.070
Kati Lassila-Perini: That on, in addition to the speakers, we have a tutorial team, we have a.

8
00:03:03.090 --> 00:03:19.170
Kati Lassila-Perini: Great facilitator, who have been helping us in testing and providing providing the material and will be there to help you help you with the exercises, so you can see here the list of people who are who are involved.

9
00:03:22.320 --> 00:03:31.320
Kati Lassila-Perini: One of the particular views of these times, obviously the remote workshop, so we would really like that we get to know each other so.

10
00:03:31.950 --> 00:03:46.170
Kati Lassila-Perini: Please tell something about your in this word called pamphlet it's a it's an APP where you can tell something about the upload your photo, if you like, and that you can also see other participants and the and organizes.

11
00:03:49.230 --> 00:03:52.080
Kati Lassila-Perini: A first level, I would like to address the goals.

12
00:03:53.520 --> 00:03:55.320
Kati Lassila-Perini: Of this workshop workshop so.

13
00:03:56.760 --> 00:04:08.460
Kati Lassila-Perini: What we think you expect from it, and what what do we actually expect from that from from to get from this workshop.

14
00:04:10.290 --> 00:04:13.530
Kati Lassila-Perini: So, first of all, we made some assumptions.

15
00:04:14.940 --> 00:04:25.590
Kati Lassila-Perini: So what we think you would like to get from the workshop is to get understanding of the basic physics object usage, that is electrons protons neutrons and jets.

16
00:04:26.730 --> 00:04:31.680
Kati Lassila-Perini: Also, the event selection and triggers that were active during the day to taking.

17
00:04:33.630 --> 00:04:45.240
Kati Lassila-Perini: We assume want to know the luminosity evaluation, which is important if you want to make analysis of your own, and also the possibilities for for processing data in larger scale.

18
00:04:47.550 --> 00:05:05.430
Kati Lassila-Perini: Obviously, we think that you will also be interested how have you put all this together in a in an analysis flow and also you properly hope to get some guidance, what is needed to bring this to the research level quality which we obviously not be able to do in in just four days.

19
00:05:07.320 --> 00:05:15.510
Kati Lassila-Perini: So this is not all, so we want to get something from you as well, so what we want from from CES and organize this is the data providers.

20
00:05:15.780 --> 00:05:25.560
Kati Lassila-Perini: We would like to get to build a community of users would like to remind you that we have feedback channels, such as the open data forum which I gave the link here.

21
00:05:26.910 --> 00:05:36.480
Kati Lassila-Perini: important for us, we also want to get an understanding of the of your usage patterns of your your needs and get feedback from you of.

22
00:05:37.050 --> 00:05:51.120
Kati Lassila-Perini: What is missing in the in the documentation and this tutorial material that you will work through, and we will also use this material to build a proper cms open data user guide I will give you a link afterwards.

23
00:05:54.240 --> 00:05:58.470
Kati Lassila-Perini: These obviously ambitious skills, so the question is, do we reach them.

24
00:05:59.550 --> 00:06:15.660
Kati Lassila-Perini: And I would like to ask you bear with us this is yes, this is our second workshop of this kind and we've certainly use solid experience that we got from last year's workshop to to improve, but this is still work working brokers.

25
00:06:17.970 --> 00:06:25.770
Kati Lassila-Perini: Now, how to get to the skills, so I will say key words about the workshop structure and now working methods so.

26
00:06:26.790 --> 00:06:41.130
Kati Lassila-Perini: When we built the material in this workshops, we thought this is what are the ingredients of a physics analysis and we built the program thinking of you planning your own analysis so.

27
00:06:42.150 --> 00:06:48.270
Kati Lassila-Perini: We want to answer to the certain questions so want to give you some knowledge, for example, how the data.

28
00:06:49.050 --> 00:07:09.750
Kati Lassila-Perini: To be recorded were selected, what are the basic steps required to get access and identify the physics objects in interests of your analysis from these open data data sets you will also learn a typical stepwise analysis workflows so as you see in this schematic figure.

29
00:07:12.060 --> 00:07:20.130
Kati Lassila-Perini: To the to the right we normally we have few steps one step, the first step is always how to get the data out of the box, how to.

30
00:07:20.520 --> 00:07:33.300
Kati Lassila-Perini: Get the data you interested in out of the public data sets so normally would write this data in a smaller format and then at the next steps you would analyze that informant.

31
00:07:34.800 --> 00:07:43.530
Kati Lassila-Perini: Obviously, I have to say at this point that, please note that the full research level analysis will take you much more time and effort them, then these four days.

32
00:07:43.770 --> 00:07:56.220
Kati Lassila-Perini: So I would like to invite you to read this blog which i'm linking which you can find in the in my slides in the agenda to get an inside of experience of people who have been working.

33
00:07:57.300 --> 00:07:58.530
Kati Lassila-Perini: would seem a certain date.

34
00:08:01.080 --> 00:08:09.420
Kati Lassila-Perini: As you've seen most of you saying there's a set of mandatory pre exercises yeah that many of you have already already gone through.

35
00:08:11.130 --> 00:08:18.030
Kati Lassila-Perini: These exercises are important like to set up and test that you really have the working environment before the workshop.

36
00:08:18.360 --> 00:08:27.000
Kati Lassila-Perini: Because we know that downloading it takes a little bit of time, and you will also need to get used to it if you're new to this to this technologies.

37
00:08:27.690 --> 00:08:42.180
Kati Lassila-Perini: And it's also some of the pre exercises where that to give you some background information on the topics that will will address it, this workshops and many thanks for your questions, we hope that we have answered most of them and to help you get started.

38
00:08:44.370 --> 00:08:51.540
Kati Lassila-Perini: Just a word of warning that if you didn't do the The pre exercise is that if you haven't seen one of this figures.

39
00:08:52.440 --> 00:08:58.080
Kati Lassila-Perini: Before you may be in trouble if you try to follow, because these really.

40
00:08:58.650 --> 00:09:09.810
Kati Lassila-Perini: Are the environment that you can't use, for example, for certain users, a word of warning you won't be able to use your usual X plus account to go through the these exercises you really need to have.

41
00:09:10.260 --> 00:09:24.780
Kati Lassila-Perini: The environment which is compatible with the cms open data if you haven't done the pre exercise, please try to catch up, we do our best to help you, but obviously we need to give the priority to the ongoing exercises.

42
00:09:26.760 --> 00:09:38.820
Kati Lassila-Perini: The tutorial workflow we have identified different set of lessons, so we have lesson and data sets and access we have lessons and basic physics objects.

43
00:09:39.390 --> 00:09:52.920
Kati Lassila-Perini: And we have lessons on advanced use of this physics physics objects which we then combine to 11 about analysis flow and, finally, how to do the analysis in a cloud environment.

44
00:09:55.770 --> 00:10:09.090
Kati Lassila-Perini: This the schedule is available in the indycar agenda but there's also a link to a document, where you have this color coding, so we have full full days of work ahead of us.

45
00:10:10.680 --> 00:10:28.590
Kati Lassila-Perini: You may notice that the The agenda is only in the afternoons, but we expect that you do the online offline work on on the exercises then during your morning so evenings depending on your time zone.

46
00:10:31.620 --> 00:10:44.580
Kati Lassila-Perini: about the working methods, you may have noticed that we have indicated the lessons with with colors so it's easy for you to follow follow the agenda, because as we've divided the lessons in different.

47
00:10:45.990 --> 00:11:00.750
Kati Lassila-Perini: Different parts, some of these different parts of it can be considered bit scattered in agenda, but you can find them if you follow the colors so each lesson is divided either pre pre exercise which you already done or a proper to work.

48
00:11:01.890 --> 00:11:02.220
Like.

49
00:11:03.450 --> 00:11:06.930
Kati Lassila-Perini: Then we have a DEMO system and questions.

50
00:11:08.520 --> 00:11:16.290
Kati Lassila-Perini: Then we have hanson system, during which you will really be able to work through some exercises.

51
00:11:16.770 --> 00:11:32.250
Kati Lassila-Perini: But then the main work is done during the challenge, which is something you will do, on your own time so handsome it's during the time which is indicated in the agenda and challenge is during a challenge and the proper to work is on your time.

52
00:11:33.360 --> 00:11:42.780
Kati Lassila-Perini: Take note that each lesson has a dedicated maximus channel in the same way similar way that we had in the in the pre exercises, so please go through the most channel and.

53
00:11:43.830 --> 00:11:54.870
Kati Lassila-Perini: Ask the questions in always with the with the correct channels, so we can easily follow up and also other people can can find find your questions and the answers there.

54
00:11:57.840 --> 00:12:04.260
Kati Lassila-Perini: As a reminder, we have the same as analysis and cloud environment tutorial which will take place.

55
00:12:05.280 --> 00:12:13.800
Kati Lassila-Perini: later in the week, so you will have the opportunity to learn how to run a a seamless job in in a commercial cloud environment.

56
00:12:15.900 --> 00:12:22.200
Kati Lassila-Perini: If you want to participate it's important that you let us know through survey at the latest tomorrow.

57
00:12:23.100 --> 00:12:38.550
Kati Lassila-Perini: This is because it will be handsome and you will get a temporary account to these cloud resources which we got through the cloud brand project which we are we, I see use case, so we will be using Cuban 80s engine on the Google cloud platform.

58
00:12:41.640 --> 00:12:46.440
Kati Lassila-Perini: And then getting help during during the workshop.

59
00:12:47.670 --> 00:12:57.750
Kati Lassila-Perini: In matmos again, please choose the Channel corresponding to the listen and do not hesitate to ask that's that's that's supposed to be interactive and ask questions.

60
00:12:58.230 --> 00:13:14.670
Kati Lassila-Perini: But do check if the same question has already been asked, and if there's an answer and says some matter most to or guidance formatting guidance, so, if needed, to use the tactics for some code or a block of code.

61
00:13:15.720 --> 00:13:22.890
Kati Lassila-Perini: And if you don't if you want a new line in your in your message to shift return for a line break.

62
00:13:24.810 --> 00:13:28.110
Kati Lassila-Perini: During a tutorial if there are some.

63
00:13:29.250 --> 00:13:45.810
Kati Lassila-Perini: Something which needs to be updated, will do it so reload the tutorial page every now and then to to see those updates and during the live lessons, you can type in questions in zoom chat or use raise hand for for was questions.

64
00:13:48.810 --> 00:14:01.170
Kati Lassila-Perini: Then sister little reminder, so we are really happy that you are so many we have 137 registrations and 50 have the five of you have already gone through the pre exercises.

65
00:14:01.740 --> 00:14:16.230
Kati Lassila-Perini: At we're not actually that many to help you but luckily, we have some participants from cms who have agreed to help us along, so thank you, thank you very much for these participants, we are really, really glad that you are, you have agreed to.

66
00:14:18.030 --> 00:14:19.350
Kati Lassila-Perini: To to help us.

67
00:14:20.940 --> 00:14:24.360
Kati Lassila-Perini: When you ask help please read the instructions carefully and.

68
00:14:25.470 --> 00:14:34.560
Kati Lassila-Perini: It may happen as we are not professional technical writers so some details are actually some important parts in the structure may not be.

69
00:14:35.310 --> 00:14:48.690
Kati Lassila-Perini: As well exposed in the text that they should be, but we hope that the instructions are are are complete, so do them, step by step, every them obviously suggestions for improvements for clarity.

70
00:14:49.830 --> 00:14:51.180
Kati Lassila-Perini: I most most welcome.

71
00:14:52.740 --> 00:14:59.130
Kati Lassila-Perini: So, despite these warnings ask ask ask, so I think this is important.

72
00:15:00.720 --> 00:15:01.620
Kati Lassila-Perini: We are really.

73
00:15:03.090 --> 00:15:16.350
Kati Lassila-Perini: we're not pushing for what those who know the most we want to take care of that no one is left behind, so if you if you're new to the area it's normal that you have questions of the four other people are very, very clear so.

74
00:15:17.010 --> 00:15:20.700
Kati Lassila-Perini: Do not hesitate do not hesitate asking questions.

75
00:15:23.790 --> 00:15:30.270
Kati Lassila-Perini: Then about questions, how to get help after So what about your information, social and communication.

76
00:15:31.110 --> 00:15:38.280
Kati Lassila-Perini: So, first of all, we have the CERN open data portal which you will get very familiar today when that starts with the data sets counting.

77
00:15:38.850 --> 00:15:44.910
Kati Lassila-Perini: It serves the data and associates things and usage examples, so we have data.

78
00:15:45.330 --> 00:15:58.470
Kati Lassila-Perini: Meta data we have the environment, we have a list of certified runs we have different kinds of things, and also the examples are linked from there, but how that take into account the CERN up and data portal is primarily a data catalog.

79
00:15:59.670 --> 00:16:14.190
Kati Lassila-Perini: What we want to achieve, we want to build a proper seamless open data guide, so this is working brokers, it will be completed with the material from this tutorial and also, if you want to help you are most welcome to participate.

80
00:16:16.350 --> 00:16:24.660
Kati Lassila-Perini: Then we have soon open data for room so again during this workshop we asked you to use the matters channels.

81
00:16:25.320 --> 00:16:37.260
Kati Lassila-Perini: But then afterwards feel free to post post questions and also feel free to reply as well, and we will take care of that the most frequently asked questions at this workshop will be added to this forum.

82
00:16:39.750 --> 00:16:51.810
Kati Lassila-Perini: Then other sources of information for technical questions about the open data portal, or maybe questions to limited audience, which you don't want to put in an open open area, we have the support mail.

83
00:16:53.010 --> 00:17:10.170
Kati Lassila-Perini: Then we have cms instructions which I didn't see in this workbook and cms software guide, however, take note that these are documentation which are in using cms currently so the instruction may not correspond to the version which is needed for open data.

84
00:17:11.340 --> 00:17:30.900
Kati Lassila-Perini: And if you really want to take a deep dive, then you go to the source code and again there keep in mind the worsening for cms open data which is currently in public keys, you will use the same as deputy five three x as attack in the in the in the data.

85
00:17:33.510 --> 00:17:35.580
Kati Lassila-Perini: Then, just the words of a news and outlook.

86
00:17:36.600 --> 00:17:43.710
Kati Lassila-Perini: Just briefly remember the beta releases that we've done by now and then SMS open data plans.

87
00:17:45.000 --> 00:17:50.940
Kati Lassila-Perini: So cms has already done six data releases since 2014.

88
00:17:52.320 --> 00:18:03.360
Kati Lassila-Perini: we've started with a with a very small release form it's 2010 data and gradually we've added more things in each of this race it's not only.

89
00:18:04.920 --> 00:18:12.480
Kati Lassila-Perini: Adding new open data but it's also improving continuously improving the usability I think tools, adding.

90
00:18:13.440 --> 00:18:26.310
Kati Lassila-Perini: Adding things that make it easier to use so currently we have protein and protein and heavier data for run one and all the early data is is in public domain and also 50% of the 2000 protein production date.

91
00:18:29.310 --> 00:18:30.420
Kati Lassila-Perini: Then future releases.

92
00:18:31.470 --> 00:18:36.630
Kati Lassila-Perini: cms data policy was first written in 2012, but it was recently updated in September.

93
00:18:38.580 --> 00:18:51.210
Kati Lassila-Perini: It actually reflects the current release the delay, which is six years after beta taking so this year we will release the first data sets from 2015 that is run to.

94
00:18:53.040 --> 00:18:58.200
Kati Lassila-Perini: The new version of the policy takes also into account the slowdown of the luminous and increase so.

95
00:18:59.520 --> 00:19:07.950
Kati Lassila-Perini: We have defined that the open data will be limited to 20% of the total luminosity of the similar data, while we're still taking date.

96
00:19:08.910 --> 00:19:16.620
Kati Lassila-Perini: Have nothing changes as soon as continuous regular data releases and the next day to release is will include first friend to data.

97
00:19:17.400 --> 00:19:34.980
Kati Lassila-Perini: Which is will be in the smaller format which we call meaning ad and it will be followed by the run to data from 2016 and then later 2017 and 18 which will also be made available in the Ad formats, with some of you may have heard.

98
00:19:36.450 --> 00:19:47.550
Kati Lassila-Perini: And as you may have heard as a common CERN open data policy which was announced in in December so we're really glad to see that other experiments are joining this effort as well.

99
00:19:48.840 --> 00:19:57.510
Kati Lassila-Perini: So now let's go to work enjoy the workshop and we'd love to hear feedback from you on Thursday.

100
00:19:58.710 --> 00:20:03.510
Kati Lassila-Perini: And i'm ready to answer questions if he if you have any.

101
00:20:37.980 --> 00:20:40.230
Kati Lassila-Perini: If no questions they would go directly to work.

102
00:20:46.920 --> 00:20:50.040
Edgar Fernando Carrera Jarrin: Okay let's let's stop the recording right.