Core Concepts
Efficiently deploying scientific workflows in HPC environments with provenance data capture using containerization strategies.
Abstract
The content introduces ProvDeploy, a framework for configuring containers for scientific workflows with integrated provenance data capture. It discusses challenges in deploying workflows in HPC environments and evaluates different containerization strategies using DenseED on SDumont CPUs and GPUs. The study explores the impact of strategies on execution time, CPU consumption, and performance.
Directory:
- Abstract
- Introduction
- Background and Related Work
- Containerization Principles
- Provenance Services
- Related Work Overview
- ProvDeploy: Assisting the Deployment of Containerized Scientific Workflows in HPC Environments
- Architecture of ProvDeploy
- ProvDeploy in Action
- Case Study: DenseED
- Environment Setup
- Exploring Different Containerization Strategies
- Conclusion
- Acknowledgments
- References
Stats
"SDumont is a cluster with an installed processing capacity of around 5.1 Petaflop/s."
"Average CPU consumption for DenseED is 55% for the coarse-grained strategy."
"In GPUs, there is no statistical difference between the presented strategies."