An Architecture for Privacy-Preserving and Replicable High-Recall Retrieval Experiments
| Title | An Architecture for Privacy-Preserving and Replicable High-Recall Retrieval Experiments |
| Publication Type | Conference Paper |
| Year of Publication | 2016 |
| Authors | Roegiest, Adam, Cormack, Gordon V. |
| Conference Name | Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval |
| Publisher | ACM |
| Conference Location | New York, NY, USA |
| ISBN Number | 978-1-4503-4069-4 |
| Keywords | experimentation, high recall, privacy, pubcrawl, rest, security, virtual machine, virtual machine security, virtualization privacy |
| Abstract | We demonstrate the infrastructure used in the TREC 2015 Total Recall track to facilitate controlled simulation of "assessor in the loop" high-recall retrieval experimentation. The implementation and corresponding design decisions are presented for this platform. This includes the necessary considerations to ensure that experiments are privacy-preserving when using test collections that cannot be distributed. Furthermore, we describe the use of virtual machines as a means of system submission in order to to promote replicable experiments while also ensuring the security of system developers and data providers. |
| URL | http://doi.acm.org/10.1145/2911451.2911456 |
| DOI | 10.1145/2911451.2911456 |
| Citation Key | roegiest_architecture_2016 |