Speaker
Cyril L'Orphelin
(CNRS/CCIN2P3)
Description
Lavoisier is an Open Source Tool developed at CC-IN2P3 used in the project EGEE/ LCG by the Operations Portal. Lavoisier is an extensible service designed to provide an unified view of data collected from multiple heterogeneous data sources (e.g. database, Web Service,LDAP, text files, XML files...). This unified view is represented as XML documents, which can be queried using standard languages, such as XSLT (and soon: XPath, XQuery).
The access to data views can be done through different standard ways : SOAP, REST/JSON or REST/XML.
Lavoisier also improves performance by caching the generated data views. The caching mechanism can be tuned according to the characteristics of the data views and of the data sources, and according to the constraints of the use-cases. This tuning has no impact, neither on plug-ins, nor on user's code.
The global architecture has four layers: the legacy data sources, the adapters used to generate the data views, the cache mechanism with flexible refresh triggering rules, and the service exposing both SOAP and REST (with XML or JSON)
interfaces.
Moreover the Lavoisier architecture permits a clear separation of roles and changes related to a role have no impact on other roles :
- the plug-in developer adds support for new technologies by developing adapters.
- the service administrator configures the adaters and cache management according to the characteristics of the data views and data sources, and according to the constraints of the use-cases.
- the user queries the data views.
This last year an effort has been put to ease the integration of data for service adminstrators by developing web interfaces which permits to :
- see the list of views and their characteristics
- see the status of these views and the eventual errors linked with these views
- see the configuration file
- evaluate the dependencies between views .
Lavoisier has proven effective in increasing maintainability of the Operations portal, by making its code independent from the technologies used by the data sources and from the data cache management policy. Its design and interfaces made easier writing reusable code, and good performances are easily obtained by tuning the cache mechanisms in an absolutely transparent way for the portal code.
Indeed, the different components work in a standardized way through the output of the Lavoisier Web Service. The translation of resource information into this standardized output is provided by different plug-ins.
Recently, a huge joint effort has been recently put into the configuration of Lavoisier, the structure of its caches and the rules of refreshing to have efficient, scalable and reliable data handling. Indeed, all information has been structured around the base component of the Operational Model: the site. We retrieve the global information about primary sources like GGUS (official ticketing system for EGEE) , GOC DB (official database for sites in EGEE), SAM (Monitoring System) , or Nagios (another Monitoring system) and we organize it by sites.
The main idea is to construct a summary of the different available information for a site: firstly, this organization permits to continue to work with the caches, even if a primary source is unavailable; then you access only information you need on the web page. Your information is structured around a synoptic view of the site and you don't access hundreds of times the primary sources but a subset of them with the site view.
Finally, we refresh the data sources only when we need it and only when an action has been triggered.
Last but not least, it is very easy to add a new data source in this model. In the configuration file of Lavoisier you add the access to the primary source and also the split of this information per site.
This information is then readily available in the synoptic view of the site.
Primary author
Cyril L'Orphelin
(CNRS/CCIN2P3)
Co-author
Mr
Sylvain Reynaud
(CNRS/CCIN2P3)
