Jul 3 – 5, 2006
CERN, Geneva
Europe/Zurich timezone

Pyphant - A Python framework for modelling reusable data processing tasks

Jul 5, 2006, 9:00 AM
30m
40-SS-D01 (CERN, Geneva)

40-SS-D01

CERN, Geneva

Refereed Papers Refereed Papers

Speaker

Mr Klaus Zimmermann (Freiburger Materialforschungszentrum, Universität Freiburg)

Description

We present the Open Source framework Pyphant for the creation and application of data flow models. The central idea of this approach is to encapsulate each data processing step in one unit which we call a worker. A worker receives input via sockets and provides the results of its data processing via plugs. These can be inserted into other worker's sockets. The resulting directed graph is called a recipe. Classes for these objects comprise the Pyphant core. To implement actual processing steps Pyphant relies on third party plug-ins which extend the basic worker class and can be distributed in so-called Pyphant-worker-archives (PWA). On top of the core, Pyphant offers a data exchange layer based on scipy arrays and PIL images which facilitates the interoperability of the workers. A third layer comprises textual and graphical user interfaces. The latter allows for the interactive construction of recipes, the former for the batch processing of data. Our contribution discusses the Pyphant framework and presents example recipes for certain scientific data analysis tasks taken from image processing and the solution of ill-posed problems.

Primary author

Mr Klaus Zimmermann (Freiburger Materialforschungszentrum, Universität Freiburg)

Co-authors

Dr Andreas W. Liehr (Freiburger Materialforschungszentrum, Universität Freiburg) Mr Lorenz Quack (Freiburger Materialforschungszentrum, Universität Freiburg)

Presentation materials