PanDA RUCIO integration plan meeting

Thursday 10 Jul 2014, 16:00 → 17:00 Europe/Zurich

61/1-017 - Room D (CERN)

Participants: Alessandro Di Girolamo, Vicent Garonne, Cedric Serfon, Felix Lee, Kuo-Hao Ho, Chen-Hsi Chao, Kuashik De, Tadashi Maeno, David Cameron.

ASGC (Felix Lee):
 * Two numbers are here in CERN to join PanDA and RUCIO team for 6 months.
 * In Taiwan, AMS and other applications need panda brokage badly.
 * We need to get PanDA+RUCIO brokage working by Christmas.

RUCIO( Cedric ):
 * The new dq2client compatible release will be ready next week.
 * PanDA+JEDI could use all dq2 client api to call rucio transparently, so, PanDA-JEDI doesn't need any changes.
 
Atlas( Alessandro ):
 * Already had some discussions with RUCIO team, as of now, the RUCIO dq2 compatible client is alsmost there, so, the migration from dq2 to RUCIO could be transparent for Atlas.
 * The only thing has to do is to have some tests to validate RUCIO dq2client functionality.
  - Comment from Vicent: it would be good to have seperate panda server for AMS, so, AMS can do test directly with new dq2client, if everything is going fine, then we can move to next step to test new dq2client for Atlas.
  - Comment from Alessandro: PanDA had the same setup for CMS before, so, it's also feasible for AMS, only needs to tell pilot to contact specific panda server.
  - Comment from Tadashi: It's feasible to setup different panda server by puppet for AMS.

 * Next step, possibly, Atlas has to live with two backends during the integration test.  
  - Maybe to start with HC analysis test, because the work flow is much simpler, and then make particular task to use RUCIO backend.
   - Question: How can JEDI be aware of two backends, can we define different backend for specific activity in JEDI?
   - Comment from Vicent: RUCIO could offer extra attribute with registration api to indicate which backend to use while job registers output file and datataset.
   - Comment from Tadashi: It should be no problem to impletment specific backend for certain tasks or specific analysis users.
   - Comment from Alessandro: If JEDI can use different backend for certain task, it will big step for panda-jedi RUCIO integration, because whenever we suffer error, we can do fix accordingly and then repeat the test again and again until everything is fine.
 
Action items:
 * Tadashi:
  - Give a list about what PanDA is doing with dq2client to Cedric
  - Add new task parameters into JEDI task, so that, PanDA-JEDI can switch to different backends by task basis.
 * Cedric:
  - Once received panda-dq2 calls list from Tadashi, Cedric can validate new rucio dq2client to make sure it doesn't miss anything and then make proper new dq2client release.
 * Kuo-Hao:
  - Need to setup panda-JDEI for AMS and then also do test and validation with new dq2client.
  - Can also help Atlas test if feasible.

16:00 → 17:00 Drafting plan

To draft plan for PanDA RUCIO integration