Metrics Meeting Minutes ----------------------- Attended: Eamonn Kenny, Gianni Puccini, Jozef Cernak Please note, the discussion wasn't in the order specified below. Immediate Concerns ------------------ * discussion about the granuality of metrics. * EK: middleware metrics or PT metrics? * GP: different granulity for different metrics. For coding metrics might be better to have granularity at the PT but have bug-tracking metrics per middleware in some cases. Action Point: Take metric granularity on a case by case basis. Product team granularity should be the norm. * EK: Bug-tracking metrics have very different granularity in Savannah compared with RT (DESY) and bugzilla (ARC). Action Point: include recommendations on how to improve tracking systems, per product team, severity and detection area of the certification process. Should come into line with Savannah. Also, recommend eventual cohesion between bug-tracking systems (only a recommendation, we have no power to enforce this). Metrics Summary from PTs ------------------------ * EK: states that Valgrind can be erroneous even if the code works perfectly. * GP: asks what threading tests provides. * EK: says needs more investigation. * GP: says deadlock checks would be good. Action Point: EK to look into threading and valgrind metrics over next few days. May need to split into more metrics. Metrics will be added to internal document. * EK: we need to specify what tools are available to provide unit tests, jUnit, cppunit, pyUnit, etc. Is there a cunit test? We need to be clear on what is available. * GP: we are not so interested in unit test metrics, but instead in coverage tests for unit tests. * EK: all metrics can be fudged, so we will need SA2.5 to monitor this with spot checks. Action Point: GP will expand out the coverage test description and add a list of tools used. (in addition check for perl, bash tests) Metrics summary from DSA2.1 --------------------------- * EK: states that it would be very good if the documentation metrics could be automated using a repository such as: ProductTeamName/Date/Release/DocNumber or something that can be interrogated using ETICS. * GP: can we really do this? * EK: most likely not, but we can produce a recommendation to help streamline the end process. (not mentioned in meeting, but worth noting: streamlining is a large part of goal 1 of the project) Action Point: include recommendations about how to streamline or introduce metrics, especially in the case of different * EK: SA2.4 may have a list of metrics they would like to obtain from the repositories that they have introduced. If so, we need to know about them soon? Action Point: GP to ask Lorenzo and Andres for information, in light of Reviews section of DSA2.1, which includes per platform, per format (tgz/rpm/deb), per change/patch, per external granularity requirements. Metrics Layout -------------- * Discussion about what is missing from metric layout. Action point: Both EK and GP will add risk and goals sections to any metrics produced before the next (first agile) meeting on Monday 12th July 2010. Assignment of Tasks ------------------- * GP: suggests we include state-of-the-art section in document * EK: suggests we move appendices including metrics into a chapter to keep things in line with the progression laid out by ISO25000 standard. Action Point: EK will rearrange document to include Introduction, state-of-the-art, metrics surveys, architecture, metrics, references, background reading Action Point: GP will add goal and risk specifically to coverage metric, and will include where possible metrics based on EGEE-III NA1 report Action Point: GP to send link to EK: http://project-egee-iii-na1-qa.web.cern.ch/project-egee-iii-na1-qa/EGEE-III/QoS/Follow-up/Middleware/Middleware.htm (already done, straight after meeting) Action Point: Next meetings: 12th, 13th, 15th, 16th July 2010, at 10:30CEST (all 4 meetings are going to be agile meetings to discuss internal document).