Task Force on Applications
Task force leader: Mateo Valero, BSC
General Objectives
HIPEAC has a unique strength in bringing together research on architectures, programming models, compilers and tools. In order to use this strength wisely, we want to go against the tide and connect software research to architectural research using applications and algorithms as drivers. We will promote some -carefully selected, realistic and representative of user domains- applications to be open to the HiPEAC community, the EU funded projects and also to the public domain.Our goal is to increase source code sharing and collaboration within the HiPEAC community, in order to provide a common set of applications to evaluate new research ideas. A concrete effort that all HiPEAC clusters will benefit from is to define a benchmark suite composed by a set of well characterized applications and input data sets.
Due to the HiPEAC organization in clusters, one of the most crosscutting topics is the need for representative applications on top of which we can show our research progress in a way that can be compared to actual technologies, reproduced in multiple sites, and discussed among academia and industry. We would like to agree on an applications-driven research agenda.
To promote a real impact on all clusters, we look forward to have at least one active contact person at each of the nine HiPEAC clusters. Our Task Force will periodically meet, coordinate and organize activities to further develop activities on promotion and dissemination of high quality benchmarks for new computer architectures.
During the time span of the Task Force, we would like to have enough momentum to be able to cover (and have enough member’s teams to focus on) scientific parallel applications, industrial high-performance applications, as well as applications for embedded systems, for low power systems, for real time, etc . We know that good application classification is crucial to our success, for there is a workload convergence in benchmark candidates, meaning that they share a common set of mathematical models and techniques, and further down, similar numerical algorithms and primitives.
Being a Network of Excellence, we want to nurture the EU HPC ecosystem and especially companies that are active in the area. We are aware that new technologies enable new applications, and that we have to look at new user requirements. We want to give a first class status to new metrics like: scalable performance, programmability, compile/optimization time, power efficiency, etc. But we also want to address what really matters to users, like response time. That’s why we will be involved in the development of the HiPEAC Roadmap. In fact, it will allow us to double check which areas are considered important by the Roadmap and the Task Force.
As an example, we consider that benchmarking is needed to evaluate new research solutions on heterogeneous architectures. There is a debate if new programming models are needed, how compilers have to deal with multiple ISA applications, and how loaders and runtime/OS have to manage the heterogeneity of these new systems. Benchmark suites for such heterogeneous architectures (like the Cell BE or the GPU platforms) are missing and we don’t know any other body that is trying to build them.
We all have experienced that porting applications to heterogeneous CMP is a time-consuming and error-prone task. To optimize them and get expected speedups has only been done by expert architects and programmers for very few applications. Those codes are rarely shared. We would like to change this behavior, and encourage people to devote time to clean up their codes, make them available to Task Force first, which will act as a discussion forum on the appropriateness of the code to become a benchmark, to be ported to other platforms, to validate realistic input data sets, and to agree on a set of experiments that can be reproduced and compared. Then, after a voting process, we will promote some of the most representative benchmarks to be open to the HiPEAC community, the EU funded projects and also to the public domain. This final selection can be guided by a Task Force Board, composed by the representatives of the nine HiPEAC clusters, the more active members in the Task Force, the industry representatives, and the Task Force Chair.
Work plan
During the duration of the project, the work plan will accomodate the contributing members interests. As a starting point of possible tasks that we could plan to achieve:- Identify the set of Application Domains that can benefit from this effort and decide which ones we can deal with. Look at realistic applications from financial, science, medical, educational, consumer, etc environments.
- Identify Programming Models and Computer Architectures that are relevant to this study in our community
- Select the platforms we are going to test taking into account the features that are significant: multi-core processors, heterogeneity, computation scalability, memory hierarchy, application domain accelerators, communication and synchronization mechanisms, network-on-chip, power constraints, etc
- Define and establish a consensus in our field, the adequate metrics to evaluate new platforms that have made obsolete the speedup only previous measurements.
- Create groups in charge of encouraging the port of applications to different programming models and to different platforms
- Promote a repository of applications, where HiPEAC members contribute, download and use project management and bug tracking tools
- Start de the discussions at the Task Force Board to select applications that fulfill some quality criteria and are representative of an area of knowledge
- Promote the set of selected applications to become benchmarks suites supported by HiPEAC in order to use them in our collaborative research and publications.
- Be proud of and disseminate the “EU HiPEAC Benchmark Suite for ..." (programming model, platform, domain…)
- Participate at the HiPEAC Roadmap meetings and contribute to the writing.
- Regular conference call meetings of the Task Technical Applications Board
- Task Force on Applications meetings at the HiPEAC Computing Systems Weeks and HiPEAC Conference.
- Hands on Tutorial for PhD. Students and feedback session from benchmark users at the HiPEAC Summer School.
- Organize tutorials and promote participation at the International Workshop on OpenMP (IWOMP), the Programmability Issues for Multi-Core Computers (MULTIPROG) Workshop, and other similar conferences and events that our task force members organize.
Link to Repository and Forum
- Increase source code sharing and collaboration within HiPEAC community
- Provide a common set of applications to evaluate new research ideas
- Define a benchmark suite composed by a set of characterized applications and input data sets
- Provide a portable/extensible framework to set up experiments and retrieve experimental results
- Establish a discussion forum to get opinions about the status of the applications, the requirements to run on a specific platform, the applicability to a domain, the accuracy of the performance results, etc
- Setup the voting and promotion mechanisms to select the applications that will be part of the Benchmark Suites.
- The web page of the task-force as well as a mailing-list is up. The task-force has presently more than twenty registered members
