PTB Meeting

Tuesday 9 Nov 2010, 14:00 → 16:00 Europe/Zurich

Teleconference

PTB meeting, 14:00-16:00, 9 November, 2010 Calling details: skype test conferencing service in Lund (call mailmagnes in skype) Screensharing: https://my.dimdim.com/arc-emi (12345) Proposed agenda: ---------------- - Approval of previous meeting notes, follow up on actions - Logging and Bookeeping service: area change? - Job wrapper - Task forces: quick check - Compute area architecture for EMI Final - AoB

text LB_non-compute_use_cases.txt

PTB meeting, 14:00-16:11, 9 November, 2010

Prsesent: Marco, Laurence, Morris, Balazs (notes), Patrick (left at 15:24)
Excused: John


1) Approval of previous meeting notes, follow up on actions

All previous actions were completed. Meeting notes from 26/10/2010 are approved.


2) Logging and Bookeeping service:  area change?

PTB continued the discussion from last meeting in view of the L&B use cases posted by Zdenek. Balazs's summary of the L&B use case mail: 

Czech team considers L&B as a generic monitoring service with the following capabilities:
* event gathering
* event/message delivery, message "concentration"
* message archive (storing messages even for quite a long term)
* providing non-trivial processing
* and exposing the processed results

From the current EMI software stack the L&B is capable collecting events from 
WMS and CREAM. Not clear to me what it can do with the SEs.

The L&B team presented couple of use cases in their mail:
1) job status monitor for WMS, CREAM
2) CMS Experiment Dashboard using the RTM that in turn utilizes the L&B
3) logging of sandbox file transfers, first step towards recording generic data 
transfers. This use case came from StoRM developers.
4)  L&B is extensively used by Grid Observatory to "analyze behaviour of grid 
components not just individual jobs"
5) foreseen usage within EGI operation. Unclear details due to Tiziana's holiday.
6) L&B as an alternative messaging solution based on "the very mature, stable, 
and reliable event delivery mechanism of L&B". The L&B event delivery chain is seen as a potential tool to 
deliver custom messages into the messaging infrastructure
7)  L&B as a syslog record transmitter.
8) L&B as a Certificate Revocation List update propagator.

The infra area leader's response on the use cases: 

1) In production, however, it only available for the two gLite services 
above. This raises an important harmonization issue within EMI.

2) This is again uses the Job Status information, however, I am trying to 
contact  Julia Andreeva to understand the priorities.

3) Is this in production? What are/who can make use of this feature? Why is 
it required? Sandbox file transfers is a Compute Area issue

4) The Grid Observatory, it is not the primary customer of EMI. They have a 
use case that they wish us to support, however, they have mentioned that 
putting events in a messaging infrastructure would be the best way to 
support them.

5) No concrete use cases specified. EMI should gather use cases for it's 
customers.

6) This is in conflict with the direction of EMI. The EMI Messaging PT will 
provide the messaging solution.


7 and 8 ) not an EMI use case


Further discussions:

Laurence: compute use case is ok, though L&B need to cover non-glite CEs as well.
The non-compute area requirements are not really strong.  The posted use cases were not convincing, most of them fall into a low-priority category (from EMI point of view).

Morris: A generic L&B service, as a event collecting/processing/archiving solution is not a top priority for EMI.

Marco: Two separate messaging would be a bad design.

Laurence: he prefers to see the L&B focused on the compute functionality. 

Morris: The Czech team should focus on compute area, before the L&B considers covering yet unclear other area use cases.

Laurence: the proposed generic event archive is not part of EMI workplan. That use case doesn't exist.

Patrick: He seconds Laurence's arguments. 

Balazs: Mentions that monitoring is investgated by a dedicated task force within EMI-JRA1, the guys from the L&B PT are welcome to join as individual emi colleagues but not as the "L&B PT".

Marco: he states that has no strong disaggrement. Sees no big difference whether the L&B belongs to infra or compute area.

Laurence: currently, the most relevant area for L&B is compute, because that is the one with well-understood and justified use cases.


PTB voting about the question: "Do you support moving the L&B producvt team to the compute area"?
Yes: Laurence, Patrick, Morris, Balazs
No: Marco 

Decision: The L&B product team is moved to the compute area from the infrastructure one. This will enable the L&B team to better focus on their highest priority use cases originating from the compute area. 

Longer justification:

In the context of EMI, the current use cases supported by the L&B product falls within the Computing Area. The L&B product team have aspirations to evolve the L&B product into a generic event logging and processing tool. Although such a direction may be interesting to EMI, at this point in time it is not a priority and there are no clear use cases provided by our customers. The priority for EMI is to adopt a messaging infrastructure and the focus of the L&B PT should be to ensure that the existing use cases are supported in this way. The move to the Computing Area, will ensure that this is the focus and help with harmonization within the Computing Area. If in the future we find the need for a generic event logging and processing tool, we can consider how to address this with EMI and evalute the L&B server as a potential solution. 


A note concerning delivreables:  the first year area workplans need not to be changed. The area change will be reflected in the EMI wiki and the 2nd release of the deliverables.


4) Task forces: quick check 

The PTB went through the task forces that were created on the 3th and 4th August PTB meetings:

* Virtualization TF

The task force showed no activity so far, does not attend infra meetings. Peter Solagna is the leader, the TF has a team defned. Nevertheless some initial plan will be presented at the All-hands.

* Cloud TF

In good shape. Shahbaz acts as  "de facto" leader. Cloud Workshop at All hands.

* Accounting TF

Leader John Gordon. TF is in good hands. Members are OK. No meeting yet. Talk at All Hands.

* Service monitoring TF:

The task force was a black whole so far, nothing happened.  Danilo Dongiovanni got proposed as a leader (to be confirmed).  The TF should be populated with one person per consortia as a begining.


* Remote service management TF. 

The scope of this task force was not 100% understood. Jason was the only person involved in this task force. He worked from the Unicore perpective. Need to find additional people from other consortia.

The PTB reinforces that this task force is about remotely managing services: stop, start, optionally remotely modifying configuration.

* EMI does not have a configuration task force, see previous EMI PTB decision related to configuration layer.

* Task force related note: EMI Registry is a PT, not a task force.

5) Compute area architecture for EMI Final

postponed to next meeting due to shortage of time

6) AoB

* next meeting: three weeks from now, after the all-hands, unless there is something urgent comes up in connection with all-hands. That case there'll be a meeting next week Tueday morning.

* DNA1.3.1 status. Balazs has no plans to make major changes.

* architecture figure: Balazs will send some comments about the figure. The new figure is not that bad. Meanwhile, Marco is working on an example for compute area using graphviz.


ACTIONS:
-----------
Balazs adds job-wrapper to the component list in DNA1.3.1, Marco sends the necessary data including package names.