Visible to the public Biblio

Filters: Keyword is reproducibility  [Clear All Filters]
2022-02-25
Wittek, Kevin, Wittek, Neslihan, Lawton, James, Dohndorf, Iryna, Weinert, Alexander, Ionita, Andrei.  2021.  A Blockchain-Based Approach to Provenance and Reproducibility in Research Workflows. 2021 IEEE International Conference on Blockchain and Cryptocurrency (ICBC). :1–6.
The traditional Proof of Existence blockchain service on the Bitcoin network can be used to verify the existence of any research data at a specific point of time, and to validate the data integrity, without revealing its content. Several variants of the blockchain service exist to certify the existence of data relying on cryptographic fingerprinting, thus enabling an efficient verification of the authenticity of such certifications. However, nowadays research data is continuously changing and being modified through different processing steps in most scientific research workflows such that certifications of individual data objects seem to be constantly outdated in this setting. This paper describes how the blockchain and distributed ledger technology can be used to form a new certification model, that captures the research process as a whole in a more meaningful way, including the description of the used data through its different stages and the associated computational pipeline, code for analysis and the experimental design. The scientific blockchain infrastructure bloxberg, together with a deep learning based analysis from the behavioral science field are used to show the applicability of the approach.
2019-10-30
Belkin, Maxim, Haas, Roland, Arnold, Galen Wesley, Leong, Hon Wai, Huerta, Eliu A., Lesny, David, Neubauer, Mark.  2018.  Container Solutions for HPC Systems: A Case Study of Using Shifter on Blue Waters. Proceedings of the Practice and Experience on Advanced Research Computing. :43:1-43:8.

Software container solutions have revolutionized application development approaches by enabling lightweight platform abstractions within the so-called "containers." Several solutions are being actively developed in attempts to bring the benefits of containers to high-performance computing systems with their stringent security demands on the one hand and fundamental resource sharing requirements on the other. In this paper, we discuss the benefits and short-comings of such solutions when deployed on real HPC systems and applied to production scientific applications. We highlight use cases that are either enabled by or significantly benefit from such solutions. We discuss the efforts by HPC system administrators and support staff to support users of these type of workloads on HPC systems not initially designed with these workloads in mind focusing on NCSA's Blue Waters system.

2017-12-12
That, D. H. T., Fils, G., Yuan, Z., Malik, T..  2017.  Sciunits: Reusable Research Objects. 2017 IEEE 13th International Conference on e-Science (e-Science). :374–383.

Science is conducted collaboratively, often requiring knowledge sharing about computational experiments. When experiments include only datasets, they can be shared using Uniform Resource Identifiers (URIs) or Digital Object Identifiers (DOIs). An experiment, however, seldom includes only datasets, but more often includes software, its past execution, provenance, and associated documentation. The Research Object has recently emerged as a comprehensive and systematic method for aggregation and identification of diverse elements of computational experiments. While a necessary method, mere aggregation is not sufficient for the sharing of computational experiments. Other users must be able to easily recompute on these shared research objects. In this paper, we present the sciunit, a reusable research object in which aggregated content is recomputable. We describe a Git-like client that efficiently creates, stores, and repeats sciunits. We show through analysis that sciunits repeat computational experiments with minimal storage and processing overhead. Finally, we provide an overview of sharing and reproducible cyberinfrastructure based on sciunits gaining adoption in the domain of geosciences.

2017-09-26
Papadopoulos, Georgios Z., Gallais, Antoine, Schreiner, Guillaume, Noël, Thomas.  2016.  Importance of Repeatable Setups for Reproducible Experimental Results in IoT. Proceedings of the 13th ACM Symposium on Performance Evaluation of Wireless Ad Hoc, Sensor, & Ubiquitous Networks. :51–59.

Performance analysis of newly designed solutions is essential for efficient Internet of Things and Wireless Sensor Network (WSN) deployments. Simulation and experimental evaluation practices are vital steps for the development process of protocols and applications for wireless technologies. Nowadays, the new solutions can be tested at a very large scale over both simulators and testbeds. In this paper, we first discuss the importance of repeatable experimental setups for reproducible performance evaluation results. To this aim, we present FIT IoT-LAB, a very large-scale and experimental testbed, i.e., consists of 2769 low-power wireless devices and 127 mobile robots. We then demonstrate through a number of experiments conducted on FIT IoT-LAB testbed, how to conduct meaningful experiments under real-world conditions. Finally, we discuss to what extent results obtained from experiments could be considered as scientific, i.e., reproducible by the community.