SA2.6 Meeting: RELEASE POLICY, deployment, verification

Tuesday 2 Aug 2011, 14:30 → 16:00 Europe/Zurich

CERN

• Feedback on proposed changes for Release Management policy
  • amount of time for deployment in the process
  • immediate releases
  • trackers and similar
• Order of Verification/Deployment on testbed
  

  MARIA's Proposal
  "
  "Certified" -> "Ready for Testbed" -> "Testbed Deployed" -> "Verified"
  
  Where the details of the workflow are:
  
  - STATUS CERTIFIED:
  QC checks the Dashboard. They check that at least, Deployment and Basic Functionality tests are OK:
    - If yes, the task is moved to "Ready for Testbed". In case other things are missing in the cert report, QC indicates it in the task. PTs can provide a new cert report in the upcoming days.
    - If not, the task is moved to "Open" and reassigned to the relevant PT.
  
  - STATUS READY FOR TESTBED:
  Cristina populates the relevant repositories and creates a GGUS ticket requesting the testbed to deploy the new version of the product. The Testbed team deploys the product and runs the tests. Once this is done, two things can happen:
    - Danilo/Tomasz change the task state to "Testbed deployed" and the field "Testbed Result" is updated with PASS.
    - Danilo/Tomasz change the task state to "Open" and the field "Testbed Result" is updated with FAIL. The task is reassigned to the relevant PT.
  - Danilo/Tomasz close the relevant GGUS ticket.
  
  - STATUS TESTBED DEPLOYED:
  QC checks the Dashboard and generates the verification report, attaching it to the task. The task is moved to "Verified"."
  

• does it still make sense to have RC + prod testbed ?
• Intercomponent testing: what do we do until Nagios?
• release notes: apply update without specifying what to do on other components... 
• Shoudl we have a place to note issues which do not seem worth a ggus tkt?
• AOB

Connection details:

Title:        EMI SA2.6: Release Management policy
Description:    
Community:    Universe
Password:    emisa26

Meeting Access Information:
- Meeting URL
    http://evo.caltech.edu/evoNext/koala.jnlp?meeting=M2MvMB2e2sDaDi9u9nDM92

- Password: emisa26
- Phone Bridge
    ID: 377 6977
    Password: 6598

Central European Summer Time (+0200)
    Start    2011-08-02  14:30
    End      2011-08-02  15:30

Participants: Andrea C. . , Danilo D. , Bjorn H., Cristina A., Giuseppe F., Maria A.P., Alberto A., Tomas W.

• Order of Verification/Deployment on testbed

Approved workflow:
"Certified" -> "Ready for Testbed" -> "Testbed Deployed" -> "Verified"

In detail:

"Certified" -> "Ready for Testbed" IF 1st QC verification check pass (1)

"Certified" -> "Open" IF 1st QC verification check fails (2)

"Ready for Testbed" -> "Testbed Deployed" IF Testbed PASS

"Ready for Testbed" -> "Open" IF Testbed FAIL (3)

"Testbed Deployed" -> "Verified" AFTER 2nd QC verification check (4)


(1) 1st QC Verification check should use "Production Release Criteria" and every mandatory criteria is OK.
(2) 1st QC Verification check should use "Production Release Criteria" and any of the mandatory criteria fails.
(3) If we can´t deploy the new product in the testbed, the task goes back to open.
(4) No matter whether there are things missing (they are minor, otherwise the task was moved to Open before), the task is moved to verified with the necessary remarks added in the report.

Summary of actions:
-- Cristina to review and give feedback on "Production Release Criteria" https://twiki.cern.ch/twiki/bin/view/EMI/ProductionReleaseCriteria
-- Cristina to update Savannah with:
   -- "Ready for Testbed" state.
   -- Update transitions so that the following workflow is possible:
      -- "Certified" -> "Ready for Testbed" -> "Testbed Deployed" -> "Verified"
      -- "Certified" -> "Open"
      -- "Ready for Testbed" -> "Open"
   -- Notifications to be added:
      -- "Certified" : Giuseppe, Claudio, Michele, Maria, Jozef and Andrea.
      -- "Ready for Testbed" : Cristina
      -- "Testbed Deployed" : Giuseppe, Claudio, Michele, Maria, Jozef and Andrea.
      -- "Verified" : Cristina
-- Maria to update:
   -- "Change Management Policy" and "Check list" with new states.
   -- "Production Release Criteria" with Cristina's feedback.

Notes
- everything from verification that can stop deployment on testbed should be done before testbed deployment
- Testbed deployment time can be used from PT to fullfill some missing points in verification (but nothing that requires re-deploying on testbed)
- ACTION (Cristina) minimal prioritize requirements for emi release criteria
- ACTION (? ) create "Ready for testbed" state, notification to Cristina

•  REPOSITORIES:
  • deployment repository created aligned with the GGUS tkt deployment request
• Amount of time for deployment in the process
  • 1 week startign from ggus tkt by cristina collecting components to deploy
• immediate releases:
  • will stop normal updates
  • repository just with what necessary
• trackers and similar
  •   stop notifications to TESTBED, just through GGUS
• does it still make sense to have RC + prod testbed ?
  • YES
  • keep two testbed sets, 1 pointing to production repo, 1 pointing to deployment (RC)
  • ACTION (SA2.6) separate testbeds in twiki, nagios and repo
  • 2 testbed updates (1 at GGUS, 1 after 1 week on production testbed) for each 1 emi update
  • do we need these 2 infrastructures up at the same time? - it depends on who and how is doing intercomponent testing (NAgios--> then 2 parallel infrastructures needed), manual (then one can switch on and off separate VMs)
  • ACTION Alberto --->HW/ Effort problem to report to PEB-->
• Intercomponent testing:
  • Missing step in the quality control of release process mentioning Nagios checks.
  • ACTION (Maria)--> ADD it in the Policy
  • ---> NAGIOS monitoring RC testbed.
    • ACTION (DANILO) ask about ARC Nagios Situation
    • ACTION (Tomasz) --> fix configuration problems of RC NAGIOS
    • ACTION (Bjorn) -->UNICORE configuration of nagios probes
• release notes: updates for libraries without specifying what to do on other components...
  • follow ggus tkt or ask RM for clarifications
• Shoudl SA2.6 have a place to note issues which do not seem worth a ggus tkt? 
  • NO, report throught GGUS tkt or ask PTs for clarifications