WEBVTT 1 00:00:02.370 --> 00:00:05.940 Matthew Bellis: Okay, I think it's recording now so thanks katia for that. 2 00:00:06.899 --> 00:00:16.770 Matthew Bellis: Excellent really, really great overview of this workshop and the goals of the workshop, as well as the goals of all the participants right, you are here to learn more about data. 3 00:00:17.520 --> 00:00:27.750 Matthew Bellis: Perhaps to discover new physics and the data set that we, the cms analysts may have missed i'm very excited for this workshop in general. 4 00:00:28.560 --> 00:00:42.000 Matthew Bellis: it's a really fascinating experiment really that cms is doing, making all of these data available to anyone in the world, I think it's been a big success to this point, and I think it will continue to be a success, going forward. 5 00:00:42.480 --> 00:00:46.050 Matthew Bellis: I comment on a couple of things quickly organizational stuff. 6 00:00:46.800 --> 00:01:00.210 Matthew Bellis: There is a link to paddle it in both matter most in the town Square, as well as in the zoom chat right now, so if you haven't had a chance to check it out, you can see both your. 7 00:01:00.720 --> 00:01:09.720 Matthew Bellis: facilitators for this, and some of your peers, some of the other participants, you can add a photo or just some text about yourself if you want. 8 00:01:10.800 --> 00:01:14.580 Matthew Bellis: Okay, I am going to talk to you about. 9 00:01:15.930 --> 00:01:29.430 Matthew Bellis: How to find your data and, first let me tell you how to find some of the materials i'm going to be sharing with you, so let me go ahead and share my. 10 00:01:30.510 --> 00:01:35.190 Matthew Bellis: See I don't think I can share my window, so this will be let me go ahead here. 11 00:01:36.210 --> 00:01:39.660 Matthew Bellis: So let me show you where i'm going to go i'm going to start from. 12 00:01:42.690 --> 00:01:55.260 Matthew Bellis: Oh, so this is the pad let me show you guys really quick, this is where you'll be able to add your pictures, as well as the pictures of some of your peers, this was not actually the window, I wanted to share but that's okay. 13 00:01:57.030 --> 00:01:58.680 Matthew Bellis: I can show you that okay. 14 00:02:00.720 --> 00:02:07.140 Matthew Bellis: What I did want to show you, you would think I would know zoom by now, after all this time. 15 00:02:09.000 --> 00:02:09.660 Matthew Bellis: Here we go. 16 00:02:11.130 --> 00:02:17.820 Matthew Bellis: Great, this is what I wanted to show you so most of you are familiar with this page right here. 17 00:02:19.650 --> 00:02:27.750 Matthew Bellis: The way i'm going to get to what i'm about to show you is, if you want to you can go to one of two places, you can go to the scientific Program. 18 00:02:30.030 --> 00:02:32.040 Matthew Bellis: Sorry, you can go to the timetable. 19 00:02:34.200 --> 00:02:42.690 Matthew Bellis: If you go to the timetable, you can see a breakdown of everything that will be happening today here is copies talk, as well as her slides. 20 00:02:43.920 --> 00:02:52.260 Matthew Bellis: And then here's the material that I will be going through with you for the next 45 minutes before setting you loose on some hands on exercises of your own. 21 00:02:53.190 --> 00:03:11.280 Matthew Bellis: There was a link directly to the lesson itself, but there are also some slides here and i'm actually going to start with the slides before we come back to the actual exercise, and this is how you can you can use this to jump to most of the lessons, if you want to. 22 00:03:12.570 --> 00:03:21.990 Matthew Bellis: Just to again point out something else if you go to the main workshop page, there is a link here tutorial site. 23 00:03:23.100 --> 00:03:34.350 Matthew Bellis: And if you click on this link for the tutorial site, you should always be able to get back to this, this is the website for all of the different exercises. 24 00:03:35.280 --> 00:03:45.780 Matthew Bellis: Some of that you've done already, as well as all of the exercises you'll be doing a content a screenshot of some of the schedules for each day. 25 00:03:46.320 --> 00:03:58.530 Matthew Bellis: This is uncertain time, although I realized that many of you may not be in Central European time and you can always get to this page again from the indigo. 26 00:03:59.850 --> 00:04:06.030 Matthew Bellis: So i'm just going to go back to our original indigo page and this link here tutorial site. 27 00:04:07.320 --> 00:04:08.700 Matthew Bellis: will take you to our list. 28 00:04:09.990 --> 00:04:13.590 Matthew Bellis: But i'm going to get to what we're going to do by going to timetable. 29 00:04:15.210 --> 00:04:26.850 Matthew Bellis: You can follow, along with this as well and there's two links here one is the data set scouting link and the one is an overview of Google slides and that's what i'm going to start with right now. 30 00:04:28.860 --> 00:04:34.620 Matthew Bellis: Just to give you a quick overview of what we're going to be doing, and this will also provide kind of a a. 31 00:04:35.670 --> 00:04:41.640 Matthew Bellis: An introduction really it's not just what i'll be doing, but what many of the other instructors will also do. 32 00:04:42.900 --> 00:04:47.550 Matthew Bellis: So this will just take a few minutes and then we'll get into the actual material itself. 33 00:04:49.650 --> 00:05:07.290 Matthew Bellis: So the way, many of these lessons are structured is there are three parts to the lessons The first one is going to be an intro or an overview and what we're going to do is we're actually going to walk through the documentation that is posted for the tutorial. 34 00:05:08.460 --> 00:05:19.380 Matthew Bellis: For the lesson that I will be going through with you there's about I can't remember if there's three or four lessons, this is an active lecture and therefore you should try to do this with me. 35 00:05:20.820 --> 00:05:29.940 Matthew Bellis: All of the information that i'll be going through with you is on the web page and I recognize it can be difficult, sometimes to both follow, along with a lecture but do it. 36 00:05:30.390 --> 00:05:41.100 Matthew Bellis: as well, I will try to slow down and make sure that you are able to follow, but you should use zoom to raise your hand, you can post in matter most as well. 37 00:05:41.850 --> 00:05:52.410 Matthew Bellis: There is a channel for this exercise data set access and one of my fellow parts of facilitators may be able to jump in one of your peers may be able to help you as well. 38 00:05:52.830 --> 00:06:02.100 Matthew Bellis: So let me emphasize, right now, all of the participants, you are in this together, and you should feel free to help each other out with all of these exercises. 39 00:06:03.030 --> 00:06:17.790 Matthew Bellis: This overview will mostly be clicking through parts of the open data portal, so it should be relatively straightforward to go through, but of course this still applies for some of my colleagues lessons as well we'll be typing out code. 40 00:06:19.230 --> 00:06:30.540 Matthew Bellis: So this will be about 45 minutes and then we'll have we have a break built into the schedule for coffee tea whatever you want water use the restroom. 41 00:06:31.710 --> 00:06:48.870 Matthew Bellis: And then it'll be a hands on activity that we've broken that we've dedicated 40 minutes to and you'll be working on this on your own, in other words there's no lecture associated with this now all the facilitators will still be here, there is still work to be done after this. 42 00:06:50.340 --> 00:06:58.770 Matthew Bellis: But you can post post questions on matter most you could raise your hand and zoom if you wanted, and if you did not understand everything in the intro DEMO. 43 00:06:59.340 --> 00:07:05.070 Matthew Bellis: Which is most likely because we're going to be introducing a lot of information kind of fast. 44 00:07:05.790 --> 00:07:11.970 Matthew Bellis: Just remember the learning really takes place when you're doing the hands on activities, this is the. 45 00:07:12.600 --> 00:07:27.150 Matthew Bellis: This is the point of this exercise of the workshop really is by doing the work yourself, is where you learn, so if some if you had some questions about the intro DEMO, that is fine, when you get to the hands on ideally that's where some of this will stick. 46 00:07:28.230 --> 00:07:43.380 Matthew Bellis: And then, after the end of the day, or end of the day's presentations again, no matter what your time zone is there's a challenge for you to work on again on your own, in other words, you will work on this asynchronously the facility facilitators will not be online. 47 00:07:44.460 --> 00:07:56.790 Matthew Bellis: And this will be today, after the scheduled lectures when I say online we won't be on zoom However, we will be monitoring matter most you should really be working, however, together, if you will. 48 00:07:57.810 --> 00:08:08.310 Matthew Bellis: With your peers, in other words, trying to help each other answer questions, some of you may be in an environment where you have a local mentor maybe a postdoc or more senior researcher that you're working with. 49 00:08:08.610 --> 00:08:21.990 Matthew Bellis: And if they help you answer some of this, you should feel comfortable posting your solutions to your peers, so that they can help you out and the solutions for this will be presented tomorrow I believe it's, the first thing on the schedule. 50 00:08:23.040 --> 00:08:36.690 Matthew Bellis: For tuesday's so this this kind of a rhythm, if you will, for an exercise will be duplicated by my by my colleagues for the other parts of this of this workshop. 51 00:08:38.430 --> 00:08:38.790 Okay. 52 00:08:40.470 --> 00:08:47.970 Matthew Bellis: So let me tell you the point of this exercise right why, why are, why are we going through, why did we put together this particular lesson. 53 00:08:48.870 --> 00:09:05.460 Matthew Bellis: So suppose you are a theorist a phenomenology just or just a very bright experimentalists or not even any of those just some amateur scientist who's been thinking about physics and you have positive a new symmetry in nature. 54 00:09:06.570 --> 00:09:18.900 Matthew Bellis: And if there's a symmetry in nature, we know that there must be some carrier for this interaction and you're like haha I know how to find this new particle i'm going to go look in the data. 55 00:09:19.350 --> 00:09:30.210 Matthew Bellis: And what does that even mean to look in the data well you're going to go to the open data portal, and you have some questions that you want to answer me move this out of the way here. 56 00:09:32.370 --> 00:09:40.680 Matthew Bellis: There, so you have some questions and one of them is well what data set should I use cms has released a few different data sets. 57 00:09:41.190 --> 00:09:51.390 Matthew Bellis: Is there more than one data set that is applicable to your search how much data are there are there only 100 events are there 1000 events. 58 00:09:52.080 --> 00:10:06.180 Matthew Bellis: Are there a quadrillion events well it's probably somewhere in the middle there what triggers have been used, which is best for my search ideally everybody has gone through the trigger pre exercise so you're aware of what I mean by triggers this pre selection of data. 59 00:10:07.380 --> 00:10:23.310 Matthew Bellis: So what data sets what triggers have been applied are any of them even good for your search and then you know that there's going to be some background So is there Monte Carlo for sm for standard model background, you have poked you have theorized something beyond the standard model. 60 00:10:24.720 --> 00:10:26.910 Matthew Bellis: And is there Monte Carlo for your signal. 61 00:10:29.640 --> 00:10:35.070 Matthew Bellis: And this is what we're going to work walk through his town to try to figure out all this from the open data portal. 62 00:10:36.510 --> 00:10:37.110 Matthew Bellis: So. 63 00:10:39.090 --> 00:10:51.120 Matthew Bellis: What we will do for the beginning is just figure out how you find any data or Monte Carlo of interest, and so the first three quarters of this exercise, you will just follow, along with me as we navigate through the website. 64 00:10:52.620 --> 00:11:01.590 Matthew Bellis: I will be going through the tutorial the screenshots are all there on the website so that you can see whether or not you are clicking on the right thing. 65 00:11:02.250 --> 00:11:10.680 Matthew Bellis: But you should click through the links as well, and if you haven't already you can open up the open data.cern.ch and, of course. 66 00:11:10.980 --> 00:11:20.970 Matthew Bellis: feel free to ask questions, either with the zoom raise hand, or you can post them either in the chat or matter most and one of the other facilitators will try to jump in and help you. 67 00:11:23.220 --> 00:11:31.590 Matthew Bellis: The last part of the activity will actually be using some of the tools to look in some of these files and see what's in them. 68 00:11:33.090 --> 00:11:52.080 Matthew Bellis: So to do that, you will need your vm virtual machine or docker image, so if you want to you can even get that going now or you know, while i'm talking and wrapping this up in another window in preparation for again this last part of the exercise. 69 00:11:56.040 --> 00:12:06.960 Matthew Bellis: What i'm not going to cover just in the interest of time is the CERN open data client the slides here have a link to the documentation for this. 70 00:12:07.890 --> 00:12:14.880 Matthew Bellis: I will just read from their explanation, it is a command line tool to facilitate downloading files from the CERN open data portal. 71 00:12:15.450 --> 00:12:22.080 Matthew Bellis: The tool enables you to query data sets hosted on the open data portal and to download and verify the individual file sets. 72 00:12:22.560 --> 00:12:34.560 Matthew Bellis: And so, as you scale up your analysis, you may find this useful not to download all the data, but to download subsets of the data if you're testing out some set of code, if you want to examine. 73 00:12:34.980 --> 00:12:50.070 Matthew Bellis: Some of the final records more closely so again, while I am not covering this today, I would like to point out that this is available for people who wants to go further, again in just exploring the data and finding out where things are. 74 00:12:52.560 --> 00:13:03.810 Matthew Bellis: Okay, we are about to get started um I will pause for a moment to see if anybody has any questions. 75 00:13:05.550 --> 00:13:06.600 Matthew Bellis: about any of them. 76 00:13:15.570 --> 00:13:17.040 Any questions. 77 00:13:20.040 --> 00:13:26.010 Matthew Bellis: All right, then let us jump right in and let's see if I can figure out how to. 78 00:13:27.840 --> 00:13:29.760 Matthew Bellis: Share properly again. 79 00:13:38.220 --> 00:13:56.520 Matthew Bellis: let's see if this is selected the correct way no yeah so i'm back on the timetable for today and i'm going to go to this link data set scouting, this is the one link that we will use for both this intro DEMO as well as for the hands on activities. 80 00:13:59.190 --> 00:14:09.090 Matthew Bellis: hands on activities and the the exercise that you will do afterwards, let me move this out of the way okay. 81 00:14:09.840 --> 00:14:21.330 Matthew Bellis: So here's what we're going to try to do we're going to try to figure out how to get you and all of your great ideas to the data, where you can test these these most awesome ideas that you have. 82 00:14:22.680 --> 00:14:31.710 Matthew Bellis: The setup here if you click on it, I won't go through it now, but basically says, you need an Internet connection, as well as docker or virtual machine setup. 83 00:14:32.340 --> 00:14:38.610 Matthew Bellis: For the last part of the exercise, I will go through an introduction i'll go through a little bit more about the. 84 00:14:39.600 --> 00:14:46.140 Matthew Bellis: Open data portal, some of the metadata talk about where the data sets what data and Monte Carlo available. 85 00:14:46.710 --> 00:14:56.640 Matthew Bellis: And then, what is in these data files and then we will have a break about 15 minutes if we are on time less if we're not on time and then the hands on activity. 86 00:14:57.570 --> 00:15:10.860 Matthew Bellis: You will do, while we are here, you will do that today, before we go on to the next X activities and then the offline challenge will do, on your own I hope they're I hope they're fun they're relatively basic. 87 00:15:12.390 --> 00:15:14.520 Matthew Bellis: But I hope it'll be fun for you. 88 00:15:15.600 --> 00:15:25.470 Matthew Bellis: Okay, so let's get started in the introduction, ideally, everybody is following along with this as well, and you may have the. 89 00:15:26.760 --> 00:15:33.720 Matthew Bellis: The open data open in some window as well, I think I have it as one of these, but i'll figure that out later. 90 00:15:34.830 --> 00:15:47.070 Matthew Bellis: Okay, so again if anybody wants to you can even jump ahead to episode four just to remind yourself of how to open your docker or vm and you can have that reading. 91 00:15:48.390 --> 00:15:57.120 Matthew Bellis: Okay, so here's The point of this exercise you've got a great idea i'm going to make this a little bit bigger, just in case anybody is following along with the screen. 92 00:15:58.020 --> 00:15:59.460 Matthew Bellis: You need to figure out. 93 00:16:00.180 --> 00:16:13.350 Matthew Bellis: What data, there is that you can work with so, in other words, you want to figure out what year was that data taken at in general, at least for run one run to there is a set of years that correspond to seven and eight TV collisions. 94 00:16:13.590 --> 00:16:22.050 Matthew Bellis: And some data that correspond to 13 TV collisions and you want to find out what triggers were applied in other words, how was the data pre selected. 95 00:16:22.710 --> 00:16:34.770 Matthew Bellis: There may not be a perfect trigger for your search, so you would like to find the one, two, maybe three triggers that provide pretty good coverage for the signal venture looking for. 96 00:16:35.460 --> 00:16:42.270 Matthew Bellis: And then, beyond the data, of course, you need some Monte Carlo you need some Monte Carlo data sets that you can use to check for background. 97 00:16:43.140 --> 00:16:55.050 Matthew Bellis: That may mimic your signal and you may want to look to see if there's any Monte Carlo that was produced for your signal and if there isn't you can work with the maintainer of the open data portal. 98 00:16:55.710 --> 00:17:06.750 Matthew Bellis: To either simulate your own data and do the full processing or perhaps even work with members of cms to produce that data of a Madison extra set. 99 00:17:07.890 --> 00:17:23.610 Matthew Bellis: Okay, so we are going to walk through this process, there is more texts than I really want you to read through right now, but I do hope that, especially as you're doing the homework you take time to go through the text and language that was written. 100 00:17:25.560 --> 00:17:32.370 Matthew Bellis: Now, on the open data portal it's not just about making data available to you, but we also want to make the data sizable. 101 00:17:32.820 --> 00:17:38.550 Matthew Bellis: And so there is some language here just to kind of explain to everybody, how you can cite. 102 00:17:39.270 --> 00:17:48.840 Matthew Bellis: specific data sets if you want, even before we get into the data sets I think it's important so that it can you can formally acknowledge the work of. 103 00:17:49.620 --> 00:17:59.490 Matthew Bellis: Many, many, many hundreds and thousands of people who work together to collect this data and so all the data sets have something called a digital object identifier. 104 00:18:00.540 --> 00:18:09.900 Matthew Bellis: There are links, if you would like to read more about this, but what it do I allows you to do is assign a unique ID to any piece. 105 00:18:10.620 --> 00:18:23.370 Matthew Bellis: of digital media, now that could be a book that could be a video that could be some music or in our case software packages or data sets and this allows someone to cite that. 106 00:18:24.480 --> 00:18:26.580 Matthew Bellis: That data set or software, if you want. 107 00:18:28.230 --> 00:18:40.680 Matthew Bellis: And so it once you we start getting into this and clicking on the links to actually look at the data sets, for example, this is a link to the double electronic primary data set and an eod format. 108 00:18:41.310 --> 00:18:48.900 Matthew Bellis: We will talk about what this means, but if you look on the page, you will eventually see that there is a dry. 109 00:18:50.190 --> 00:18:51.450 Matthew Bellis: fairly complicated. 110 00:18:52.680 --> 00:18:56.970 Matthew Bellis: grouping of numbers and letters attached with each record. 111 00:18:58.170 --> 00:19:13.890 Matthew Bellis: And this means that you can cite this if you want to you could either reference the do I, on its own or if your user of fit tech there are tools like do I to fib there is a link here to them i'll click on it. 112 00:19:15.660 --> 00:19:19.230 Matthew Bellis: So it's not working right now. 113 00:19:21.030 --> 00:19:22.470 Matthew Bellis: Oh, my gosh comparison. 114 00:19:25.200 --> 00:19:36.750 Matthew Bellis: Well, if you go on here, and if you enter in the do I, you cannot get the big tech entry that you could like cut and paste into your form, if you wanted. 115 00:19:39.360 --> 00:19:50.520 Matthew Bellis: If I can do this in real time without slowing things down too much, so I will go to this, I will grab this record here. 116 00:19:52.800 --> 00:19:56.040 Matthew Bellis: I will now go to do it to them. 117 00:19:57.930 --> 00:20:06.690 Matthew Bellis: Oh now it's actually here I guess it regenerated yay So you can see that this generated some late tech code that you can paste into. 118 00:20:07.650 --> 00:20:22.680 Matthew Bellis: Whatever document you're working on you don't necessarily have to cite you know everything but depending on your study it may be worth citing whatever you've used has the author, the title the publisher your. 119 00:20:27.750 --> 00:20:36.810 Matthew Bellis: There is also a term which you may or may not be familiar with as much called provenance prominence refers to the place of origin of something. 120 00:20:37.260 --> 00:20:48.270 Matthew Bellis: And so you're going to be i'm going to be exposing you to the naming scheme for many of these files and data sets and it's going to seem very, very complicated and I totally get it. 121 00:20:48.570 --> 00:20:57.330 Matthew Bellis: And I would forgive you if you were like this seems more complicated than it needs to be, but as experimental is We really need to know the history. 122 00:20:57.750 --> 00:21:04.860 Matthew Bellis: of how a data set was processed, we need to know the version of the software, the period of calibrations. 123 00:21:05.550 --> 00:21:16.770 Matthew Bellis: Otherwise, it becomes very difficult to keep track of what was process when and whether the Monte Carlo that we're using is consistent with the data set the collision data that we are analyzing. 124 00:21:18.480 --> 00:21:29.130 Matthew Bellis: And so provenance and this Meta data is built into these data files, so that we can follow thing, so we can backtrack if we needed to. 125 00:21:30.210 --> 00:21:41.760 Matthew Bellis: So this is not doin the provenance are not directly parts of maybe what you're going to be doing on a day to day basis, but it is useful to know why some of the information is on the website. 126 00:21:42.390 --> 00:21:49.830 Matthew Bellis: As well as why some of this may be, at first glance appears to be more complicated than you think it needs to be. 127 00:21:51.090 --> 00:21:51.510 Matthew Bellis: Okay. 128 00:21:52.800 --> 00:22:01.830 Matthew Bellis: let's go ahead and find our data I am now going to advance to the next lesson here which you can use either by going back to the main. 129 00:22:03.000 --> 00:22:06.660 Matthew Bellis: The main listing or just go click on this to the next one. 130 00:22:08.100 --> 00:22:08.430 Matthew Bellis: Okay. 131 00:22:09.600 --> 00:22:17.280 Matthew Bellis: Where are they actually before I move on, are there any questions does anybody want to raise a hand or have a comment or question. 132 00:22:28.560 --> 00:22:34.800 Matthew Bellis: All right, no problem alright so let's talk about where are the data sets let's see if we can find them. 133 00:22:35.820 --> 00:22:51.450 Matthew Bellis: Okay, so our starting point is going to be the CERN open data portal, and if you haven't opened this I would encourage you to open it now so here, I have a nice live web page that we can I can do stuff with and go through this with you. 134 00:22:53.550 --> 00:23:03.000 Matthew Bellis: back to the lesson itself so ideally everybody is on this page, this is an older page from last year, so it may have changed slightly. 135 00:23:05.100 --> 00:23:16.410 Matthew Bellis: And what we're going to do, because this is the cms open data workshop we're going to find the cms link over here and we're going to click on it to start out so under focus on. 136 00:23:17.850 --> 00:23:22.080 Matthew Bellis: I will also do that here, so I will click on cms. 137 00:23:25.260 --> 00:23:26.370 Matthew Bellis: By these links here. 138 00:23:31.650 --> 00:23:44.790 Matthew Bellis: And if you click on it, you should get something that looks like this your the the links that you see here to the different data sets are a little different because, again I did the screen capture last year. 139 00:23:45.900 --> 00:24:02.220 Matthew Bellis: But the biggest difference just if you're comparing what you've done to what I have in the screenshot is that I have collapsed these sections here and so you'll see a lot more information on yours and to keep track of things you may want to collapse. 140 00:24:04.080 --> 00:24:21.360 Matthew Bellis: These extra radio buttons here, but you can select different aspects of the data set which we will you can look at very specific documentation records information about the environment and the software and right now all we've done on the open data portal is select the cms experiment. 141 00:24:22.440 --> 00:24:27.360 Matthew Bellis: which has 3995 records, as of today. 142 00:24:28.380 --> 00:24:31.830 Matthew Bellis: Which is more than than last year at 883. 143 00:24:37.050 --> 00:24:46.470 Matthew Bellis: Now we can select only the data set rather than the software and other materials, which is what we are going to do so, going forward, you will select the data set. 144 00:24:47.100 --> 00:24:59.340 Matthew Bellis: And then eventually will just select one year there were going to be entries for the data set for 2010 2011 and 2012 and so, if you collect select data set here. 145 00:25:01.140 --> 00:25:03.330 Matthew Bellis: i'm going to read collapse these. 146 00:25:08.160 --> 00:25:08.850 Matthew Bellis: oops. 147 00:25:11.520 --> 00:25:19.500 Matthew Bellis: Again, we have filter by year on this 2010 2011 and 2012. 148 00:25:21.930 --> 00:25:33.030 Matthew Bellis: And then, as Kathy mentioned, there is coming their way there used to be some entries for 2016 that will be coming down the road. 149 00:25:33.720 --> 00:25:52.230 Matthew Bellis: So, for now, you should have cms selected as well as data set and 2012 and that's what we're going to work look for going forward so data set is selected 2012 and that's what we'll work with from here on. 150 00:25:59.730 --> 00:26:03.960 Matthew Bellis: And then you can go ahead and move on to the next lesson. 151 00:26:09.660 --> 00:26:19.470 Matthew Bellis: Okay, so I have told I have shown you absolutely pretty much almost nothing so far i've just showed you how to select the cms data which is. 152 00:26:20.190 --> 00:26:32.040 Matthew Bellis: not surprising to cms workshop, we are selected data sets right down because, again, the point of this exercise is scouting around for the proper data set we're not focusing on any of the software. 153 00:26:32.940 --> 00:26:41.340 Matthew Bellis: or other documentation, although you may at your leisure want to study some of this other information that may be very interesting to you. 154 00:26:45.300 --> 00:26:46.500 Matthew Bellis: So let's talk about. 155 00:26:48.210 --> 00:26:49.710 Matthew Bellis: What we're going to be looking at. 156 00:26:52.740 --> 00:26:59.520 Matthew Bellis: i'm going to start now using more specific terms, I should have actually had some of this in here, but I think I missed that. 157 00:27:00.750 --> 00:27:11.010 Matthew Bellis: i'm going to make a decision distinction between this should read collision data and collision data comes from the real life cms detector and then Monte Carlo data. 158 00:27:11.490 --> 00:27:24.240 Matthew Bellis: Which are simulated data that we use to perform our studies if I just say the term data and I forget to use the term collision data i'm generally referring to data which came off of the real. 159 00:27:25.440 --> 00:27:26.760 Matthew Bellis: The real cms detector. 160 00:27:28.170 --> 00:27:43.950 Matthew Bellis: Now, the main data is that we're going to be working with at least today is called run one that spans 2010 to 2012 and each one of these running periods let's say for 2010 can be broken up into some periods, a BC and so on. 161 00:27:45.000 --> 00:27:50.670 Matthew Bellis: And so you will find this in some of the data set names, you may see 2012 but then you may see it. 162 00:27:51.810 --> 00:27:56.970 Matthew Bellis: followed by a a B or C and some particular month range. 163 00:27:58.050 --> 00:28:10.020 Matthew Bellis: So again, for now, we have selected data set we've selected cms and we've selected 2012 out which now is showing the 2016 and 2018 data as well. 164 00:28:11.190 --> 00:28:15.720 Matthew Bellis: Okay, and you can see what we've selected up here at the top. 165 00:28:19.440 --> 00:28:25.170 Matthew Bellis: Your should look something like this, so it may look a little bit different in terms of the number of entries. 166 00:28:27.360 --> 00:28:38.640 Matthew Bellis: Now there are three some categories for data set collision again, as I mentioned above, but probably should have moved this earlier collision is real data that comes off the cms detector. 167 00:28:39.540 --> 00:28:50.310 Matthew Bellis: This is where you would perform your actual search for the data drive data sets our data sets that maybe have been scammed or filtered perhaps for outreach or education or. 168 00:28:51.060 --> 00:29:02.670 Matthew Bellis: I spy event display and then simulated refers to the Monte Carlo data sets the simulated data that you would use to perform studies for background or maybe sensitivity to your signal. 169 00:29:04.290 --> 00:29:24.720 Matthew Bellis: For now we're going to unselective derived and we're going to select simulation so that's all we're looking at is the collision data, this is the quote unquote real data, so if I go over here i'm going to select simulated i'm going to select derived so all I have over here is collision. 170 00:29:27.660 --> 00:29:29.430 Matthew Bellis: Right now, I show. 171 00:29:30.930 --> 00:29:33.630 Matthew Bellis: 58 entries for collision data. 172 00:29:35.010 --> 00:29:36.540 Matthew Bellis: That doesn't mean 58 events. 173 00:29:37.620 --> 00:29:39.720 Matthew Bellis: 58 it records here. 174 00:29:42.600 --> 00:29:59.220 Matthew Bellis: Now some of these data sets the names are might be a little bit confusing so we're going to look at a couple of them, the ones I happen to choose where double electron slash run 2012 B dash V one slash raw and this division of three sections. 175 00:30:00.840 --> 00:30:07.890 Matthew Bellis: divided by slashes is common to the data sets there's also one I have here single mew run. 176 00:30:09.990 --> 00:30:22.170 Matthew Bellis: Be January 2013 and you can click on these links, if you want to, if you would like to go directly to those records, these are live links, if you want to. 177 00:30:24.750 --> 00:30:33.930 Matthew Bellis: let's take a little bit of time and try to understand what these three fields refer to because you'll be working with them over the next few days of this workshop. 178 00:30:35.670 --> 00:30:37.860 Matthew Bellis: So the first field. 179 00:30:39.480 --> 00:30:45.300 Matthew Bellis: refers to the trigger and again ideally everybody has done the pre exercise. 180 00:30:46.440 --> 00:31:01.500 Matthew Bellis: But just you know if there's one takeaway if you're really, really new to this type of collider physics, what a trigger does is either at the hardware or software level is it filters out some subset of the data based on very specific criteria. 181 00:31:02.580 --> 00:31:06.420 Matthew Bellis: So, for example, if you see something that says double electron. 182 00:31:07.770 --> 00:31:20.490 Matthew Bellis: You could probably take a guess that this requires that there are at least two electrons at least two it could be three, it could be, for it could be five, but at least two electrons above a certain threshold. 183 00:31:21.780 --> 00:31:27.810 Matthew Bellis: Now, when you get into physics objects you'll find that the definition of an electron is not so simple. 184 00:31:29.310 --> 00:31:45.420 Matthew Bellis: So, for the time being, maybe it would be more appropriate to say that there was some deposition of energy somewhere in the detector that it's consistent with what one would expect from two electrons above at least some energy. 185 00:31:47.220 --> 00:31:59.430 Matthew Bellis: There is a single view on requiring at least one new on above a certain momentum threshold there's minimum bias events, these are events taken without any other sort of trigger or selection criteria, so, in other words. 186 00:32:00.450 --> 00:32:07.680 Matthew Bellis: You just open the floodgates take the data for a while it's good to test, maybe, maybe other assumption sometimes. 187 00:32:09.060 --> 00:32:12.270 Matthew Bellis: And if you have more questions about the triggers you can reach out to matter most. 188 00:32:13.560 --> 00:32:15.480 Matthew Bellis: or review the pre exercise. 189 00:32:16.710 --> 00:32:20.850 Matthew Bellis: You know there's many, many, many more triggers that we've worked with in cms. 190 00:32:21.870 --> 00:32:23.010 Matthew Bellis: Besides these three. 191 00:32:24.180 --> 00:32:25.620 Matthew Bellis: And you'll see other names here. 192 00:32:27.330 --> 00:32:52.140 Matthew Bellis: The next field is this run period so, for example, run 2012 one be version one run 2012 be 22 2013 these referred to when the data was taken and or when the data was processed, you may also see things like the one the two again version one version to have some processing. 193 00:32:53.490 --> 00:33:01.650 Matthew Bellis: This is not as important for you the analysts of the open data within cms because. 194 00:33:02.940 --> 00:33:15.000 Matthew Bellis: The data has been calibrated and process kind of on a running basis it's very important if you start your analysis at one stage with a particular set of data that you are matched up with the correct Monte Carlo. 195 00:33:16.380 --> 00:33:17.220 Matthew Bellis: Your. 196 00:33:18.390 --> 00:33:33.420 Matthew Bellis: Facility many some of the facilitators here, more so than I much more credit should be given to them for ensuring that the data that you have access to is consistent, you do not have to really keep track or make these significant choices. 197 00:33:34.620 --> 00:33:41.280 Matthew Bellis: Because it has already been made for you, and so we have really just unique run periods available. 198 00:33:42.480 --> 00:33:53.310 Matthew Bellis: for you to work with behind the scenes as a cms analysis, you have to kind of be very careful, though, because you can see, all of the different run periods, but that's what that middle field is. 199 00:33:54.540 --> 00:34:04.650 Matthew Bellis: And then the last field is the data format, you will see some sets of data that are raw, so this is information really directly from the detector. 200 00:34:05.340 --> 00:34:16.380 Matthew Bellis: In the form of hits from you know the really the detector elements and then there is the ld this is analysis object data, and this is generally where most analysts start. 201 00:34:17.460 --> 00:34:24.960 Matthew Bellis: The data has been processed in a form we're instead of saying that you have some bit of information at this detector element to this detector element. 202 00:34:25.830 --> 00:34:38.550 Matthew Bellis: You can see like some physics objects or something to work with, I think there is a hand raised, I apologize if I mispronounce your name sonny did you have a question. 203 00:34:41.670 --> 00:34:51.450 Saneev: That you are saying, if this will be when we do these on these are is some some TV, they are changing and according to the. 204 00:34:52.560 --> 00:34:59.670 Saneev: Put it that we when we present them so in open data, we will get the last one or the first one. 205 00:35:01.920 --> 00:35:03.210 Matthew Bellis: You will get the last one. 206 00:35:03.930 --> 00:35:06.000 Saneev: Okay, so most updated when we will get. 207 00:35:06.510 --> 00:35:15.480 Matthew Bellis: that's correct that's correct yeah you do not have to worry about this, but I wanted to explain what that information was there no, this has been made much easier for you. 208 00:35:16.410 --> 00:35:17.130 Saneev: Okay, thank. 209 00:35:17.550 --> 00:35:19.140 Saneev: You welcome good question. 210 00:35:22.140 --> 00:35:25.200 Matthew Bellis: Any other questions okay. 211 00:35:26.760 --> 00:35:36.360 Matthew Bellis: All right now, if you click on any of these data sets you can find even more information, like the size of the data set information on what the recommended software. 212 00:35:37.590 --> 00:35:45.000 Matthew Bellis: Information about the tech the trigger selection, as well as a list of all the individual route files. 213 00:35:45.780 --> 00:35:54.030 Matthew Bellis: And I wanted to go through that because we are actually going to use some of these actual names of these route files, if you want to inspect them. 214 00:35:54.570 --> 00:36:17.280 Matthew Bellis: manually or you will see, you will see strings of texts like this later on when you are working with some of the the objects, so let me go to that to that double electron primary set that I that I opened, actually, let me go to one of the raw ones are not the wrong ones, but the amd. 215 00:36:18.660 --> 00:36:40.200 Matthew Bellis: So this is actually the live open data portal i'm going to click on this new EG, but I could really click on any of them you'll find that deal I information but you'll also find a description of the data set this is a primary data set in a pod format all the runs from that run be period. 216 00:36:41.790 --> 00:36:53.430 Matthew Bellis: here's some information about the number of events, what is this 14,000,770 6453 events 4.4 terabytes in total that span. 217 00:36:54.990 --> 00:36:57.120 Matthew Bellis: files lot of information. 218 00:36:58.680 --> 00:37:03.090 Matthew Bellis: here's some information or information about the trigger how are these data selected. 219 00:37:04.680 --> 00:37:11.910 Matthew Bellis: And if you go to I think it's at the bottom, you can see, the list of files for any of these and so that screenshot that I had. 220 00:37:12.930 --> 00:37:16.470 Matthew Bellis: In the lesson, if you go over to list files. 221 00:37:20.160 --> 00:37:28.410 Matthew Bellis: And then actually I think if I just click download it will actually show me, and here we go here's the list of all of those different files. 222 00:37:29.100 --> 00:37:42.180 Matthew Bellis: You will learn how to use some of this information and scripts to process the data all at once, rather than typing out all 1200 and one of these these file names. 223 00:37:44.790 --> 00:37:45.210 Matthew Bellis: Okay. 224 00:37:46.830 --> 00:37:54.750 Matthew Bellis: This pre pendant route is how these files are stored and again, you will use these directory paths when you interact with some of these data. 225 00:37:56.580 --> 00:38:06.750 Matthew Bellis: Now i'm with the Monte Carlo you can go through a similar exercise with this we're going to go back through and select. 226 00:38:07.680 --> 00:38:21.690 Matthew Bellis: The collision data and instead select the simulated data and we're going to click another button at the top, which is included on demand data set so now we're going to look to see what Monte Carlo is available. 227 00:38:23.010 --> 00:38:30.360 Matthew Bellis: So i'm gonna actually let me go ahead and just start at the main page. 228 00:38:31.590 --> 00:38:33.090 Matthew Bellis: i'm going to cms. 229 00:38:34.140 --> 00:38:36.150 Matthew Bellis: i'm going to click data set. 230 00:38:39.570 --> 00:38:42.240 Matthew Bellis: i'm going to select the 2012 data. 231 00:38:46.680 --> 00:39:00.990 Matthew Bellis: And instead of looking at the collision date i'm actually going to select the collision i'm going to select arrived, and so what I have now is a list of 368 records, and these are now all Monte Carlo. 232 00:39:03.300 --> 00:39:04.410 Matthew Bellis: or simulated data. 233 00:39:07.410 --> 00:39:11.760 Matthew Bellis: By go back to the lesson again if you've selected things, you will see. 234 00:39:12.840 --> 00:39:20.790 Matthew Bellis: Maybe I didn't did I do the on demand data sets no I forgot to click this section here, so this on demand data sets now i've got. 235 00:39:22.080 --> 00:39:23.580 Matthew Bellis: just wondering why that was a bit small. 236 00:39:27.690 --> 00:39:34.380 Matthew Bellis: So you'll have cms 2012 the simulated data set and the on demand data sets. 237 00:39:36.060 --> 00:39:42.660 Matthew Bellis: Again, our data is named broken up into three fields separated by slashes. 238 00:39:44.580 --> 00:39:47.670 Matthew Bellis: The first one, instead of the run period, which is what our. 239 00:39:48.870 --> 00:39:53.100 Matthew Bellis: which is what we had before, and so the trigger was the first field in the. 240 00:39:54.120 --> 00:40:05.400 Matthew Bellis: In the other ones we have the physics process, in other words, what is this Monte Carlo sample simulating so, for example, dui jets to l l m. 241 00:40:07.710 --> 00:40:22.590 Matthew Bellis: There are links, I will show you in just a moment about how to understand this, but I can tell you that this is a trail yon process with jets to to leptons in the mass of less than 50 range. 242 00:40:23.640 --> 00:40:42.450 Matthew Bellis: Or the mass of the Di leptons pair i'm sorry supposed to be above 50 now, this can be fairly confusing all of this, but I will show you a link to learn more about this, what tuning was done what was used to produce this data set in this case it's a mad graph. 243 00:40:43.620 --> 00:40:52.740 Matthew Bellis: There is a global tag, and this is the the question before about processing periods as a cms analysis behind the scene. 244 00:40:53.070 --> 00:41:01.890 Matthew Bellis: I need to make sure that this processing information is consistent with whatever data set that i'm using you do not have to worry about that as much. 245 00:41:02.370 --> 00:41:11.340 Matthew Bellis: But for better for worse, you know we still give you the entire file name but just to explain what this middle section is it tells you something about the provenance. 246 00:41:11.970 --> 00:41:32.940 Matthew Bellis: Of when this data was processed or this Monte Carlo and under what calibration conditions now the last field for all of the simulated data is different from the collision data, and it does, all the same, it is in this case a iot Sim, so this is analysis object data simulation. 247 00:41:34.080 --> 00:41:44.520 Matthew Bellis: This means that, basically, has the same fields is what you would expect in the regular data there's some extra information associated with the fact that it is Monte Carlo data. 248 00:41:45.150 --> 00:41:59.250 Matthew Bellis: But otherwise you can still find the same yuan's the same electrons and global nuance everything you will learn about in physics objects at this stage or analysis should be almost effectively, the same as if you process, the data itself. 249 00:42:03.090 --> 00:42:17.490 Matthew Bellis: Now, if you if you scroll further down, you will see ways to select by very specific categories of physics processes so i'm going to go back now to the live. 250 00:42:18.720 --> 00:42:26.130 Matthew Bellis: Open data portal, and if you scroll down and i'm looking on the left hand side here, you get to this filter by category. 251 00:42:27.660 --> 00:42:36.390 Matthew Bellis: And now you can see their data set specific for the physics or konya beyond two generations exotica like dark matter extra dimensions rabbit on. 252 00:42:36.870 --> 00:42:44.280 Matthew Bellis: Left oh corks heavy gauge bows ons and just as important in all of that standard model of physics. 253 00:42:44.730 --> 00:42:55.350 Matthew Bellis: These are your potential backgrounds, and so we have provided you with a whole bunch of data sets simulating generic ucb talk physics, which is a background too many processes. 254 00:42:55.800 --> 00:43:09.180 Matthew Bellis: electroweak drawing you on processes, this is where you'll spend a lot of time trying to understand whether or not these set standard model backgrounds simulate your signal of interest. 255 00:43:09.810 --> 00:43:32.820 Matthew Bellis: And unfortunately, a lot of times they do, and then it's up to you, as the you know the searchers for truth, the seekers for the underlying understanding of nature, to try to figure out how to separate background and signal that is where that's where the rubber meets the road, as they say. 256 00:43:34.410 --> 00:43:43.260 Matthew Bellis: Okay, so you can go through, and you can take a look at these at your leisure, to try to explore these, and it is worth poking around all of this. 257 00:43:45.120 --> 00:43:58.290 Matthew Bellis: Okay um any questions about this before going on to the next section, ideally, you know how to, or at least you've seen how to navigate the open data portal to find some things. 258 00:43:59.550 --> 00:44:00.930 Any questions. 259 00:44:05.190 --> 00:44:07.680 Matthew Bellis: alrighty then okay moving along. 260 00:44:10.410 --> 00:44:24.780 Matthew Bellis: Okay, so what is in these files we've really seen nothing of the data so far, so what we're going to do is take a look at them, ideally, and so, hopefully, most people here have. 261 00:44:26.130 --> 00:44:34.830 Matthew Bellis: gone through the pre lessons on either virtual machines or docker you should have a working cms area or know how to get there. 262 00:44:35.550 --> 00:44:42.960 Matthew Bellis: You will see this pre I copied this blatantly from one of my colleagues lessons about. 263 00:44:43.290 --> 00:44:58.920 Matthew Bellis: How to set up your cms s w area as a reminder so here's a reminder of how to do stuff with the the virtual machine and, of course, you can go to the links for the pre exercises, if you want to for the virtual machine or docker. 264 00:45:01.980 --> 00:45:05.610 Matthew Bellis: little bit of reminder here of how to start your container. 265 00:45:08.880 --> 00:45:24.930 Matthew Bellis: And what you want to do is once you're in that environment, make sure that you are in the the source directory and then make sure you run the cms envy script and this sets up the environment, so that you have access to. 266 00:45:26.580 --> 00:45:30.510 Matthew Bellis: You have access to the cms software tools. 267 00:45:33.120 --> 00:45:40.260 Matthew Bellis: Now there is a whole suite of tools prefixed by edm this stands for event data model. 268 00:45:41.460 --> 00:45:53.610 Matthew Bellis: You can use these to dump information about individual files it's very useful to just take a quick look and see what's in them right, these are massive files. 269 00:45:54.210 --> 00:46:03.060 Matthew Bellis: tremendous amount of information and then you can get information about prominence from them, as well as just the data that's in the files. 270 00:46:04.800 --> 00:46:12.930 Matthew Bellis: And so the, the one that i'm going to just introduce you to although if you take edm and then hit tab you'll see that there are other ones. 271 00:46:13.470 --> 00:46:33.630 Matthew Bellis: The one that number of us will use on regular basis, and you may find, as well as something called edm dump event contact, I would encourage you to copy and paste the name of this or just be careful of the capitalization the camel capitalization This is sometimes referred to in programming. 272 00:46:34.680 --> 00:46:41.400 Matthew Bellis: For the first term is lowercase and then the other words dump event content or capitalized. 273 00:46:43.200 --> 00:46:57.540 Matthew Bellis: And so, if you type this out, I happen to have a session, let me see i'm going to stop share, and then I am going to see if I can share this terminal. 274 00:46:58.860 --> 00:47:00.180 Matthew Bellis: doesn't look like I can. 275 00:47:05.640 --> 00:47:06.030 Matthew Bellis: There we go. 276 00:47:12.630 --> 00:47:13.230 Matthew Bellis: Here we go. 277 00:47:16.920 --> 00:47:18.270 Matthew Bellis: stretch this out there, we go. 278 00:47:19.470 --> 00:47:21.840 Matthew Bellis: So in this environment, here is my. 279 00:47:23.520 --> 00:47:25.890 Matthew Bellis: i'm in home cms user. 280 00:47:29.760 --> 00:47:37.620 Matthew Bellis: And if I just cut and paste that command or i'm just going to try to type it out edm I can hit tab to auto complete. 281 00:47:38.700 --> 00:47:50.940 Matthew Bellis: You see there's a bunch of edm tools quitting something called prob dump prominence done but right now we're going to use edm dump event content. 282 00:47:52.380 --> 00:48:00.030 Matthew Bellis: dance that's how you'll see that there are a number of options such as print only branch names and everything. 283 00:48:05.730 --> 00:48:09.330 Matthew Bellis: And if you grab the. 284 00:48:11.760 --> 00:48:13.440 Matthew Bellis: command from the. 285 00:48:15.060 --> 00:48:15.870 Word i'm looking for me. 286 00:48:24.750 --> 00:48:38.460 Matthew Bellis: If you grab the command from the from the lesson, you see what we do is we wind up going to a particular file one of those I think I grabbed a draw yawn perhaps I know TT jets. 287 00:48:39.420 --> 00:48:56.700 Matthew Bellis: And I redirect to the output I dumped everything and instead of dumping, to the screen a dump it to a file, but I call test edm output block so all I did was I cut and pasted it i'm going to use VI to look at it, but you could use less or tab or anything else. 288 00:48:57.840 --> 00:49:12.210 Matthew Bellis: And if you run it, it may take anywhere from 10 to 30 seconds to run, you may have a lot of people hitting that particular file, but if you go in there, you find all of this information there's the lh he event products that's live. 289 00:49:13.800 --> 00:49:31.230 Matthew Bellis: Events to file format for simulated data trigger results there's a vector of gen jet objects and then you can see what they are, these are Ak five gen jets you will learn about what this all refers to, and this is just a small subset of the data. 290 00:49:32.610 --> 00:49:37.800 Matthew Bellis: You can keep scrolling through you can find this all this information about the milan's. 291 00:49:39.660 --> 00:49:41.040 Matthew Bellis: So on so on. 292 00:49:44.550 --> 00:49:54.750 Matthew Bellis: And so, if you're interested to see if a particular file has you know information, maybe you're looking for jets maybe you're looking to study photons. 293 00:49:55.860 --> 00:50:03.510 Matthew Bellis: Maybe you need some combination of them it's a quick way to see some of this stuff and see what the names of these, these are. 294 00:50:05.040 --> 00:50:12.480 Matthew Bellis: At the very end of this exercise, I have a few other commands that are useful to kind of like play around with. 295 00:50:13.050 --> 00:50:26.160 Matthew Bellis: just get a sense of what they can do if you were looking for very specific stuff one of them adds this dash dash REG ex which is regular expression searching for anything that has a new on in it. 296 00:50:27.360 --> 00:50:33.630 Matthew Bellis: may take a moment to run, but you see that I get everything with nuance here. 297 00:50:38.850 --> 00:50:42.780 Matthew Bellis: And if you run it again, I will grab the next entry. 298 00:50:45.780 --> 00:50:49.080 Matthew Bellis: In the exercise this one has a dash dash name. 299 00:50:50.160 --> 00:50:58.770 Matthew Bellis: At the beginning, and what it'll do is it'll do a little bit simpler output and just give the names of all of these fields. 300 00:51:00.060 --> 00:51:01.380 Matthew Bellis: That are in here. 301 00:51:07.440 --> 00:51:17.100 Matthew Bellis: i'm going to stop sharing i'd see that there's a couple of hands raised i'm not sure who raised their hand first so i'm just going to go in the order that they appear on my screen dylan did you have a question. 302 00:51:18.270 --> 00:51:31.500 Dillon Fitzgerald: yeah I was just curious if users have the ability to read close to their own jets using your software, if you just offer a set of different jets with different readings parameters, or something like that. 303 00:51:31.890 --> 00:51:42.000 Matthew Bellis: ya know so that's a great question, so there are ways to access the and i'll let my colleagues correct me on this, you know some of the some of the. 304 00:51:44.040 --> 00:51:51.420 Matthew Bellis: Some of the publications that have been using the cms data have actually been studies of JET substructure. 305 00:51:52.380 --> 00:52:03.210 Matthew Bellis: And so I cannot remember off the top of my head if they use the Ad information if there was enough in there for them to access and then redo the JET clustering. 306 00:52:03.840 --> 00:52:15.030 Matthew Bellis: Or if they had to go back to the raw data, but I do know that there is a way to access that information, such that you could write your own clustering algorithm if you want to. 307 00:52:15.780 --> 00:52:21.870 Matthew Bellis: And there are in fact subsets of the data that have been specifically released for people who want to do machine learning. 308 00:52:22.560 --> 00:52:35.160 Matthew Bellis: Specifically, you know, particularly for things like jet clustering with the data so that has a little bit more low level information France, I think, or dylan I think I saw on padlocked that you're a member of lh CD. 309 00:52:36.180 --> 00:52:36.900 Dillon Fitzgerald: yeah that's right. 310 00:52:37.230 --> 00:52:39.330 Matthew Bellis: yeah so um so yeah so there has been. 311 00:52:39.360 --> 00:52:40.170 Dillon Fitzgerald: You know, so I can. 312 00:52:40.230 --> 00:52:42.390 Matthew Bellis: I can use maybe a bit more specific terminology yeah. 313 00:52:43.680 --> 00:52:50.880 Matthew Bellis: I do not know how much are exactly who is use some of this data for machine learning, but it is out there. 314 00:52:51.090 --> 00:52:51.510 Dillon Fitzgerald: And it just. 315 00:52:53.160 --> 00:52:53.820 Matthew Bellis: yeah just. 316 00:52:53.970 --> 00:52:54.780 Matthew Bellis: stammer yep. 317 00:52:55.320 --> 00:52:58.800 Matthew Bellis: And julie's just posted some stuff in the chat that a God. 318 00:52:58.860 --> 00:53:11.310 Matthew Bellis: does have the particle flow candidates, so that is basically the individual particles that one would use to try to cluster so you could go back and basically redo it if you wanted to. 319 00:53:12.090 --> 00:53:19.320 Dillon Fitzgerald: Okay yeah that's sort of what I was wondering if one would be missing anything if they just requested good fast jet from the list. 320 00:53:20.490 --> 00:53:34.710 Matthew Bellis: yeah, no, no, no, so not to spoil anything right, so a God has a tremendous amount of information and it's such that, for many years, analysts mostly used a simplified format called mini iot and now over the last couple of years. 321 00:53:35.730 --> 00:53:42.870 Matthew Bellis: A nano ad format has been developed, because for most that now analysts right downstream that's All they need. 322 00:53:43.200 --> 00:53:55.800 Matthew Bellis: And I mentioned many ad and man oh God, just to emphasize the Ad format actually has a tremendous amount of information in it to do these types of these studies and variations on these parameters. 323 00:53:58.410 --> 00:53:58.950 Dillon Fitzgerald: Thanks. 324 00:53:59.250 --> 00:54:01.110 Dillon Fitzgerald: For me, I could ask one more really quick. 325 00:54:01.110 --> 00:54:01.650 Dillon Fitzgerald: question. 326 00:54:01.800 --> 00:54:04.470 Dillon Fitzgerald: Is there a mechanism for users to request. 327 00:54:04.530 --> 00:54:08.850 Dillon Fitzgerald: Monte Carlo through the open data portal email a collaborator. 328 00:54:09.390 --> 00:54:10.650 Matthew Bellis: So um. 329 00:54:10.980 --> 00:54:17.730 Matthew Bellis: So I, so I don't believe that we have a formal mechanism to request it through the portal, you can use the portal to. 330 00:54:17.730 --> 00:54:20.880 Matthew Bellis: Contact some of the organizers. 331 00:54:21.900 --> 00:54:35.100 Matthew Bellis: We have had in the past, a tutorial about how to generate your own smaller samples it looks like Edgar may have a more specific answer though than I about maybe if there's any formal mechanism. 332 00:54:36.960 --> 00:54:51.810 Edgar Fernando Carrera Jarrin: Right now that there isn't there isn't a formal mechanism, but you can produce your your your own Monte Carlo samples you send cms infrastructure, there are instructions that will be mentioned. 333 00:54:52.860 --> 00:54:58.800 Edgar Fernando Carrera Jarrin: Throughout this lessons, at least in the analysis flow exam, for example, for sure. 334 00:55:00.690 --> 00:55:08.670 Dillon Fitzgerald: I see so just so i'm understanding it one would necessarily need to request you just generate a sample yourself. 335 00:55:09.990 --> 00:55:10.410 Matthew Bellis: Yes. 336 00:55:10.470 --> 00:55:21.540 Edgar Fernando Carrera Jarrin: Right, I mean if you know if if there is a good reason for releasing some Monte Carlo sample a very good reason, and if we have that Monte Carlo sample available. 337 00:55:22.170 --> 00:55:36.780 Edgar Fernando Carrera Jarrin: Then yeah you mean you could contact the coordinators of the debut a group data, the open data group and yeah we can talk about it, but you could also produce your own. 338 00:55:38.820 --> 00:55:40.020 Dillon Fitzgerald: Okay, great Thank you so much. 339 00:55:41.100 --> 00:55:41.940 Matthew Bellis: And just to emphasize. 340 00:55:42.420 --> 00:55:43.020 Edgar Fernando Carrera Jarrin: And, of course. 341 00:55:43.200 --> 00:55:43.740 Edgar Fernando Carrera Jarrin: Certainly of. 342 00:55:44.580 --> 00:55:46.710 Edgar Fernando Carrera Jarrin: course you know this, this kind of request. 343 00:55:46.800 --> 00:55:56.010 Edgar Fernando Carrera Jarrin: Has have to go through the collaboration board of the experiment and so so on and so forth, if there was such a chance to do it. 344 00:55:59.340 --> 00:56:11.310 Matthew Bellis: yeah and just to reinforce what anchor said by producing your own editors really trying to like doing the whole like through the simulation you know from starting from some very raw data and winding up with a O D Sim. 345 00:56:11.700 --> 00:56:15.180 Matthew Bellis: That you can analyze the same as you would any of these other data samples. 346 00:56:16.140 --> 00:56:19.470 Dillon Fitzgerald: So I guess you could use whatever generator you like. 347 00:56:20.220 --> 00:56:22.500 Matthew Bellis: yeah, although it comes. 348 00:56:22.950 --> 00:56:24.270 Matthew Bellis: out at some point, it needs. 349 00:56:24.270 --> 00:56:27.480 Matthew Bellis: To be in a rush file format. 350 00:56:28.560 --> 00:56:29.370 Okay, thank you. 351 00:56:32.850 --> 00:56:35.100 Matthew Bellis: and France, you had a question as well. 352 00:56:35.910 --> 00:56:41.970 Franz Glessgen: Yes, I still do, and maybe more low level question about the format of this text file we don't. 353 00:56:43.230 --> 00:56:54.360 Franz Glessgen: First of all, its full simulation and sample so the sin Rico and high level trigger references they stand for the different steps of the reconstruction. 354 00:56:55.770 --> 00:57:04.080 Matthew Bellis: Right so so Rico yes Rico is a step in the end the reconstruction, the hl T is the High Level trigger. 355 00:57:04.140 --> 00:57:06.360 Matthew Bellis: that's just additional information. 356 00:57:06.960 --> 00:57:14.490 Matthew Bellis: That is effectively a set of flags to say right so you've got this huge Monte Carlo data set for let's say drill you on. 357 00:57:15.450 --> 00:57:22.830 Matthew Bellis: Each event has a certain probability of passing any one of like a few hundred different high level triggers, for example. 358 00:57:23.700 --> 00:57:32.250 Matthew Bellis: And so the hl T fields that you're seeing are a way for you as the analyst to say look i'm using a data set with this particular trigger. 359 00:57:32.670 --> 00:57:42.990 Matthew Bellis: I can now apply these flags, you know I can search for these slides in this data in this Monte Carlo simulated data to see if it passed those different triggers. 360 00:57:44.370 --> 00:57:51.420 Matthew Bellis: that's different from the idea of like Rico which is yes, a different stage and, like the reconstruction process of the whole data processing. 361 00:57:52.620 --> 00:57:59.580 Franz Glessgen: OK, so the difference, for instance, like a hell of a trillion I got this thanks a lot, but the difference between Sim and Rico and would be. 362 00:58:00.270 --> 00:58:10.260 Franz Glessgen: Surely artificial it's not it's not just a different steps on the reconstruction Sim would be the initial particles that you simulate in the records will be after they passed through the detector right. 363 00:58:10.560 --> 00:58:23.100 Matthew Bellis: that's correct that's correct and there's intermediate stage like digi hits where we might take the simulation right so we've got the simulation of the detector but at some point, I need to turn that into like the actual electronic signals. 364 00:58:23.490 --> 00:58:34.200 Matthew Bellis: That I would imagine are coming out of the detector itself, I take those electronic signals and pass them to Rico so yeah there are these individual stages of the reconstruction process. 365 00:58:34.830 --> 00:58:43.500 Matthew Bellis: It looks like coffee may have a bit more a bit more maybe more precise details to say about these stages, so, then I caught you did you want to to jump in. 366 00:58:44.790 --> 00:58:48.750 Kati Lassila-Perini: It Sir remark, not that much precision, but. 367 00:58:49.860 --> 00:58:52.350 Kati Lassila-Perini: take note of the different vocabulary. 368 00:58:52.590 --> 00:58:53.280 Kati Lassila-Perini: That we have. 369 00:58:53.790 --> 00:59:04.980 Kati Lassila-Perini: What we, as an experimental is referred to simulation is the simulation where the generated particles have already gone through in the in the detector set up through john. 370 00:59:07.050 --> 00:59:23.190 Kati Lassila-Perini: What often the the the theories for a simulation of what we call the generated particles so generated particles in they can be either stuff that we have, for example, to thea a built in in our same as w so. 371 00:59:23.640 --> 00:59:26.040 Franz Glessgen: it's going at the start okay yeah. 372 00:59:26.460 --> 00:59:26.850 Kati Lassila-Perini: yeah but. 373 00:59:27.270 --> 00:59:33.060 Kati Lassila-Perini: If we have other other generators, then they come in come in, so in La la he format. 374 00:59:34.230 --> 00:59:37.110 Franz Glessgen: Right and the second question actually what are these row and Sigma. 375 00:59:37.110 --> 00:59:37.590 labels. 376 00:59:40.590 --> 00:59:53.580 Kati Lassila-Perini: Labels we have few data sets which are very, very small data sets which are in raw format that's for us just to study the Read reconstruction of the of the. 377 00:59:53.760 --> 00:59:56.610 Julie Hogan: I think we're looking at the edm dumped event content. 378 00:59:56.640 --> 01:00:00.630 Franz Glessgen: yeah exactly exactly sorry I wasn't meeting room was in the Greek letter and signal so. 379 01:00:02.040 --> 01:00:07.470 Kati Lassila-Perini: Excuse me know, I do not have the screen, so I can't ask the question. 380 01:00:08.100 --> 01:00:11.640 Julie Hogan: yeah this the structure, the basic structure of the of the. 381 01:00:12.690 --> 01:00:15.960 Julie Hogan: Of the dump is is that that column isn't is a name. 382 01:00:17.490 --> 01:00:27.570 Julie Hogan: For that, for the collection row in specifically refers to the ones here so you'll see here are going to be a an average energy deposit. 383 01:00:29.400 --> 01:00:44.850 Julie Hogan: For that event we typically use it for evaluating things like like well here's a new vocabulary word pile up so all the all the extra collisions that have gone on, but that column in the in the edm dump event content is kind of giving you the name. 384 01:00:46.350 --> 01:00:47.850 Julie Hogan: Of the, the object. 385 01:00:48.420 --> 01:00:49.620 Franz Glessgen: Okay, it makes sense, thanks. 386 01:00:53.550 --> 01:01:02.010 Julie Hogan: So on the on the first question that were you were seeing, which I think was also from looking at that panel where you're seeing Rico in the last column there of the names. 387 01:01:02.160 --> 01:01:13.050 Julie Hogan: You actually can make up that name all everywhere, you see Rico in that list means that all of those things in the event were created together and some process that was named Rico. 388 01:01:14.010 --> 01:01:21.480 Julie Hogan: So so actually you can see a wide variety of names even user defined names, you could make up your own process and run it. 389 01:01:23.220 --> 01:01:23.490 Franz Glessgen: Okay. 390 01:01:26.760 --> 01:01:34.830 Matthew Bellis: yeah, and this is i'm just bringing this back up here for reference the things that both doing content referring to this Rico as well as this Sam. 391 01:01:36.060 --> 01:01:46.830 Matthew Bellis: And then Miller who's file stage that as Connie said, this is really at the generator level where you see what else he and sometimes you will see Jen as well i'm like the particle gun, if you will. 392 01:01:49.200 --> 01:01:50.520 or even at the pithy level. 393 01:01:52.650 --> 01:01:59.700 Matthew Bellis: Ah awesome These are all good questions contact, did you have anything else you wanted to add can see your hand up or. 394 01:02:02.070 --> 01:02:02.370 Kati Lassila-Perini: No. 395 01:02:03.210 --> 01:02:05.310 Matthew Bellis: No worries though it's been cool um. 396 01:02:05.640 --> 01:02:07.140 Kati Lassila-Perini: These are great questions. 397 01:02:07.530 --> 01:02:17.490 Matthew Bellis: This is awesome to hear people engaging with the stuff before I move on, which is, I think, actually for a break does anybody have any questions. 398 01:02:21.180 --> 01:02:25.800 Matthew Bellis: Okay, so the next section is a. 399 01:02:27.060 --> 01:02:27.540 Edgar Fernando Carrera Jarrin: break. 400 01:02:27.900 --> 01:02:40.110 Matthew Bellis: yep, and so we have a little break here, for I believe 15 minutes if you want to you can you can jump into the hands on session, if you will, which is really just reviewing some of the material. 401 01:02:41.550 --> 01:02:49.770 Matthew Bellis: But I think we are going to stop recording from now, some of us will log off for a while and then we will be back in about 15 minutes, I believe. 402 01:02:51.990 --> 01:02:53.010 Matthew Bellis: Thank you, everybody.