Choose timezone
Your profile timezone:
PASTE meeting
→
Europe/Zurich
2-R-014 (CERN)
2-R-014
CERN
Description
CERN, bat 2 R-014
To join the telephone conference, click here: https://audioconf.cern.ch/call/0165202
To dial in to the conference:
a. Dial +41 22 7676000
b. Enter access code 0165202
Production and PASTE Meeting, 15th December 2009, CERN
Chair: A.Tsaregorodtsev
Minutes: A.Tsaregorodtsev
CERN: F.Stagni,J.Closier,A.C.Smith,Ph.Charpentier,R.Santinelli,S.Paterson,Z.Mathe, A.Mayer
Phone: A.Tsaregorodtsev,R.Graciani,M.Sapunov,A.Zhelezov
0. Ongoing Action List
12004 - Ongoing
12165 - Roberto has attached to the Savannah the info about existing Lemon metrics and alarms
that can be defined. We now have to decide which alarms to be used adn with which actions
so that operators can take this on board (ACTION: Ricardo, Andrei). Later ( in January )
we will consider the possible DIRAC specific alarms, e.g. services misbehavior.
12383 - Federico and Roberto are working on collecting site/resource status information from various
sources, aggregating it in a unform way in the ResourceStatus Service and pubilshing it
to various detinations (SAM, Dashboards, DIRAC portal). This should provide the view
of the resources status "as seen by LHCb". This is of particular interest for the
T1 site managers. Roberto is going to conduct a survey among T1 sites about which
metrics they think are the most relevant ones.
12384 - The notifications from the ResourceStatus service are now being sent to the Notification
Service. They are quite numerous. We have to start using alarms to attract attention to
more severe problems needing intervention by shifters or by other experts on duty.
It was suggested to have a dedicated Web Portal page for displaying alarms. But
we can start with just sending alarms to diracAdmin and lhcb_prod groups. Federico
had technical problems with sending alarms in the past, this should be just solved with
the help of Adri.
12529 - Using the same storage as XXX-USER space tokens for histograms looks reasonable. However,
there is a danger that histograms will fill out all the allocated space preventing users
from using it. So, quota mechanism is necessary. This will be further investigated.
In the discussion a related problem was touched: part of the RDST and RAW data should
be kept on disk for quick access. It is not quite clear how this can be easily achieved.
Andrew will formulate a separate task for the search of a solution to this problem.
1. GangaRobot description and status (A.Mayer)
Andrew presented the tests related to the user analysis grid environment that he performs using
the GangaRobot facility. The description of the tool as well as examples of the output are
attached to the agenda page.
The output of the robot jobs is parsed and the derived site status information can be published
to the SAM database ( some parameters are published by Roberto already ). It can also be used
as a source of information for the Resource Status Service.
The tool is complementary to the SAM tests, Its future development will include getting more
site metrics included. e.g. CPU/WallClock ratios. More can be done with parseing the DaVinci
log file outputs.
2. Production issues
MC running with normal failure rate. Problems accessing large RAW files on slc4_ia32_gcc34 due to
compatibility libraries (see ELOG entry). Reconstruction productions are again running smoothly after
switching back to protocol access. The reconstruction production for new data taking conditions
Beam450GeV-VeloOpen-MagDown-Excl-R1 was launched this morning and jobs are completing successfully.
With the fix for accessing large files being part of the upcoming LHCb software release we will switch
back to downloading input files in the reprocessing of the real data (RecoToDST-05).
The number of events reported in the Bookkeeping is the Total whereas in the RAW files there is
a mixture of physics and lumi events. The latter are basically empty. To avoid confusion, only
physics event statistics should be returned and displayed by the user clients.
Still problems with data access at SARA and Lyon being investigated by the experts.
3. DIRAC Releases
DIRAC v4r21p4 patch release is in production. It unsets MALLOC_CHECK_ for running Gaudi
applications and have fixes in dirac-bookeeping-get-dq-histos command.
4. DIRAC tasks
All the DIRAC systems are running on volhcb12. This was tested with creating a test production
and running resulting production jobs. Also analysis jobs accessing data were tried out.
Ricardo reported problems with the Stager. He will investigate, most likely not properly started.
Now the Certification System should be set up on volhcb03/volhcb14. Ricardo will prepare the
corresponding installation scripts.
Federico reminded that a prototype code documentation generated with Sphinx tool is available
at http://fstagni.web.cern.ch/fstagni/DIRACDOC/contents.html . Everybody is welcome to comment
in order to take decision on the final tool for the DIRAC documentation generation.
Joel noted that it would be useful to have some reference information, e.g. IT operator phone
number to be available in the DIRAC Portal. This can added to the Help menu as one of the items
( Action : Matvey ).
Joel reported that the servers for the final DIRAC Production system are delivered. He will now
proceed with their setup. In January they will be available to host the Production System.
5. AOB
Demo session is requested at the next EGEE User Forum in April 2010
There are minutes attached to this event.
Show them.