WP2 - WP5 Workshop notes and minutes ------------------------------------ Peter Kunszt - Tuesday, June 10th 2003, CERN Agenda: see http://agenda.cern.ch/fullAgenda.php?ida=a031433 SE presentation by John Gordon ------------------------------ Questions, mainly by Ian/David from LCG. for slides, see agenda page - why have the SE catalog separate from the LRC? + historical reason - take fermilab who has a well-developed mss. how does this integrate in EDG? + easy to talk to enstore... but it's not efficient (enstore has its own disk cache) + thin control layer if there is a native gridftp interface. if you need to have a gridftp interface, it comes with a disk cache out of necessity. - blocking call is ok, no? why not keep it? + because the async call over the wide area is more robust - create : 3 independent steps. same for delete. what if any one of these steps fail? how is that treated? - still don't see how to integrate with existing MSSs. + well defined interfaces will exist. if some are not fully SRM compliant, additional coding will have to be done. - is the SE a layer on top of SRM or is it an SRM? + the SE _is_ an SRM. Discussion ---------- X SRM interaction. => SE == SRM => components of an SE may be a layer on top of an existing SRM. this is especially true for authorization mechanisms. => the SE can be either a full disk resource manager, a thin layer over an existing SRM (little extra added value) or a thicker layer if the underlying MSS is not a full SRM and needs some extra functionality - this depends on the MSS. => the aim is to make all stores (including SEs) have the SRM interface. the user (replica manager) should not care. X cachestatus and createstatus are still separate. why? => will not be - getRequestStatus in SRMv1 X names - like SRMv1 => keep old, for now, and in addition have the srm methods => have new SRMv2 methods wherever functionality is missing from v1 (like for delete = srmRm) terminology : SURL : srm://host/path SFN : host/path TURL : :///path X performance of ls => owen knows of a way to optimize this. will be put down to be done low priority. X exists call: => change replica manager to use getMetadata to check for existence. => getMetadata will return the proper SRM FileMetaData object. => on errors do SOAP Faults X delete => does not exist in SRMv1, but in SRMv2. is called SrmRm. => will be done in SE. => effort to delete file from cache is minimal. from MSS its no big deal either. 'minimal effort' => to do advisoryDelete is also minimal - subset of SrmRm. advisoryDelete frees file for the garbage collector to delete the given file from cache if necessary. difference to unPin : 'you don't care about the file, garbage collector may remove file when it wants' advisoryDelete: 'you can do garbage collection on this immediately' i.e. you put the file on top of the list of files to be garbage collected. -> difference in priority assigned by the garbage collector. X host name to be given on file operations .. why? yes: SFN or SURL will be the argument - according to SRM spec X Async calls x jens working on the plan Questions to AWG / ATF ---------------------- - How to give access to outsiders (i.e. a person in another VO). => copy it to outside the grid, re-register in other VO.. what is the priority on this? - same: VO admins should be able to expose a collection of files to another VO - how? what is the priority/requirement on this? we are not sure that this is a use case at all. Other Discussions ----------------- = phone conferences are interesting and useful. = modus operandi : 1/2 hour with wp5, 1/2 hour just wp2. = management: + which representation can be merged? + iteam and q/a group does not collaborate enough. + common q/a team rep? no. Emil Knezo: gridFTP from Castor ------------------------------- Based on EDG Globus.24 + vdt 1.1.8 Supports all EDG GridFTP clients Restriction: '..' in the path will fail. Gsincftp and other clients might have problems with CWD, CDUP VO mapping: no dynamic pool-accounts, VOMS not tried yet. problems: - gridftp exists client expects that at least 1 line is sent back over data channel. - mkdir parses path from bottom to up - constatly log in, get stat, log out. takes a long time. plan: extend possible configurations with UID-stager mapping use DNS load-balancing over many servers problems: - 'data connection open' failures through nordugrid clients for RETR - might be firewall, client timeout... questions: - how to overcome host-based authentication in the DNS load-balancing? => use the same host-cert everywhere. - Interaction with Castor SRM as it should be. - Status of Castor SRM? x with the original perl-based SRM there were several problems. x reimplementation still sees a few problems, especially with request status. - as long as a file is on disk, you can get the status of the file. - if the file has been removed, you cannot get the request any more. there is no request history. - the stager needs to be re-done anyway. Items to discuss tomorrow/rest of the week ------------------------------------------ 1. SE migration plan + RC -> RLS x issue: where the files sit at the moment - is it convenient for them to stay there? x catalog needs to generate the correct SURL. 2. GLUE Schema refinements, content, outstanding problems x directories for VO - the GlueSARoot object - missing item VOName - Root may be '/' for SE.. for all VOs - need to sort with Sergio 3. Discuss File Replication x clarify terminology (usage of srmCopy, ...) x semantics, delegation, copying of metadata.. x usage, dependencies, exposed interfaces 4. Security x authentication - how to use trustmanager, - tomcat setup, also with multiple VOs - certificates - gSOAP - gsi x multi VO support - have the same set of CAs for every VO? - config to run many VOs on the same box x delegation - limited delegation - further limit capabilities x group/role based ACL - mapping of VOMS attributes to GACL? - how far WP2 and WP5 are respectively in terms of code x encryption - when is encryption happening - where to store the key - encrypted logs? - authorization on rls, rmc..