Maestro Core is part of the Maestro middleware framework, a data- and memory-aware abstraction for workflow coupling, inter- and intra-application data exchange and redistribution. A central design goal was to enable modelling of memory and storage hierarchies to allow for reasoning about data movement and placement based on costs of moving data objects. At the same time, data objects can carry user-defined metadata. This is the case, for example, for various meteorological production workflows. Leveraging the Maestro middleware allows workflow management software to reason about the data without inspecting it, at scale.
The software is still in an early stage, but has been tested in various environments with success, in terms of both functionality and performance.
Maestro Core has been produced as part of the MAESTRO project, whose middleware framework addresses ubiquitous problems of data movement in complex memory hierarchies and at many levels of the HPC software stack. The MAESTRO consortium consists of seven partners (Jülich Supercomputing Centre, CEA, Appentra, ETH Zurich (CSCS), ECMWF, Seagate, HPE), each bringing specialist knowledge and expertise to the technical challenge.
Access Maestro Core via the Jülich GitLab
Use the Maestro Core user documentation to get started