| Title | Preservation System for Scientific Experiments in High Performance Computing: Challenges and Proposed Concept |
| Publication Type | Conference Paper |
| Year of Publication | 2019 |
| Authors | Udod, Kyryll, Kushnarenko, Volodymyr, Wesner, Stefan, Svjatnyj, Volodymyr |
| Conference Name | 2019 10th IEEE International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS) |
| Keywords | case preservation, compositionality, computing system, containerization, Containers, created experiment, future reproduction, Hardware, High performance computing, HPC, HPC computations, HPC system, Internet, knowledge based systems, Libraries, operating system, Operating systems, parallel processing, Predictive Metrics, preservation process, preservation system, pubcrawl, reproducible research, research data management, research experiments, research experiments preservation, resilience, Scientific Computing Security, scientific experiment, security of data, Servers |
| Abstract | Continuously growing amount of research experiments using High Performance Computing (HPC) leads to the questions of research data management and in particular how to preserve a scientific experiment including all related data for long term for its future reproduction. This paper covers some challenges and possible solutions related to the preservation of scientific experiments on HPC systems and represents a concept of the preservation system for HPC computations. Storage of the experiment itself with some related data is not only enough for its future reproduction, especially in the long term. For that case preservation of the whole experiment's environment (operating system, used libraries, environment variables, input data, etc.) via containerization technology (e.g. using Docker, Singularity) is proposed. This approach allows to preserve the entire environment, but is not always possible on every HPC system because of security issues. And it also leaves a question, how to deal with commercial software that was used within the experiment. As a possible solution we propose to run a preservation process outside of the computing system on the web-server and to replace all commercial software inside the created experiment's image with open source analogues that should allow future reproduction of the experiment without any legal issues. The prototype of such a system was developed, the paper provides the scheme of the system, its main features and describes the first experimental results and further research steps. |
| DOI | 10.1109/IDAACS.2019.8924459 |
| Citation Key | udod_preservation_2019 |
Preservation System for Scientific Experiments in High Performance Computing: Challenges and Proposed Concept