Export Citations
Save this search
Please login to be able to save your searches and receive alerts for new content matching your search criteria.
- research-articleOctober 2024
MorphQ++: A Reproducibility Study of Metamorphic Testing on Quantum Compilers
RENE '24: Proceedings of the 2024 Workshop on Replications and Negative ResultsPages 15–21https://doi.org/10.1145/3695750.3695823Quantum computing has been rapidly expanding, and many platforms for writing programs that can be compiled and run on quantum hardware (or simulated) are being developed. As with any compiler, transformation correctness is paramount as the machine ...
- research-articleOctober 2024
MorphQ++: A Reproducibility Study of Metamorphic Testing on Quantum Compilers
ASEW '24: Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering WorkshopsPages 8–14https://doi.org/10.1145/3691621.3694959Quantum computing has been rapidly expanding, and many platforms for writing programs that can be compiled and run on quantum hardware (or simulated) are being developed. As with any compiler, transformation correctness is paramount as the machine ...
- research-articleOctober 2024
Navigating the Landscape of Reproducible Research: A Predictive Modeling Approach
CIKM '24: Proceedings of the 33rd ACM International Conference on Information and Knowledge ManagementPages 24–33https://doi.org/10.1145/3627673.3679831The reproducibility of scientific articles is central to the advancement of science. Despite this importance, evaluating reproducibility remains challenging due to the scarcity of ground truth data. Predictive models can address this limitation by ...
Changing Lanes Toward Open Science: Openness and Transparency in Automotive User Research
- Patrick Ebel,
- Pavlo Bazilinskyy,
- Mark Colley,
- Courtney Michael Goodridge,
- Philipp Hock,
- Christian P. Janssen,
- Hauke Sandhaus,
- Aravinda Ramakrishnan Srinivasan,
- Philipp Wintersberger
AutomotiveUI '24: Proceedings of the 16th International Conference on Automotive User Interfaces and Interactive Vehicular ApplicationsPages 94–105https://doi.org/10.1145/3640792.3675730We review the state of open science and the perspectives on open data sharing within the automotive user research community. Openness and transparency are critical not only for judging the quality of empirical research, but also for accelerating ...
- research-articleSeptember 2024
EpiMDE: A-Model Driven Engineering Platform for Epidemiological Modeling
MODELS '24: Proceedings of the ACM/IEEE 27th International Conference on Model Driven Engineering Languages and SystemsPages 226–236https://doi.org/10.1145/3640310.3674104Modeling is a critical step in studying epidemics. It allows us to better understand and predict the progression of a disease, design interventions such as vaccination, and assess their impact. Current epidemics are modeled using compartmental and ...
-
- research-articleAugust 2024
Context, Composition, Automation, and Communication: The C2AC Roadmap for Modeling and Simulation
- Adelinde M Uhrmacher,
- Peter Frazier,
- Reiner Hähnle,
- Franziska Klügl,
- Fabian Lorig,
- Bertram Ludäscher,
- Laura Nenzi,
- Cristina Ruiz-Martin,
- Bernhard Rumpe,
- Claudia Szabo,
- Gabriel Wainer,
- Pia Wilsdorf
ACM Transactions on Modeling and Computer Simulation (TOMACS), Volume 34, Issue 4Article No.: 23, Pages 1–51https://doi.org/10.1145/3673226Simulation has become, in many application areas, a sine qua non. Most recently, COVID-19 has underlined the importance of simulation studies and limitations in current practices and methods. We identify four goals of methodological work for addressing ...
- research-articleAugust 2024
Evaluation of Temporal Change in IR Test Collections
ICTIR '24: Proceedings of the 2024 ACM SIGIR International Conference on Theory of Information RetrievalPages 3–13https://doi.org/10.1145/3664190.3672530Information retrieval systems have been evaluated using the Cranfield paradigm for many years. This paradigm allows a systematic, fair, and reproducible evaluation of different retrieval methods in fixed experimental environments. However, real-world ...
- research-articleJuly 2024
Sharing Software-Evolution Datasets: Practices, Challenges, and Recommendations
Proceedings of the ACM on Software Engineering (PACMSE), Volume 1, Issue FSEArticle No.: 91, Pages 2051–2074https://doi.org/10.1145/3660798Sharing research artifacts (e.g., software, data, protocols) is an immensely important topic for improving transparency, replicability, and reusability in research, and has recently gained more and more traction in software engineering. For instance, ...
- research-articleJuly 2024
An Empirical Analysis on Multi-turn Conversational Recommender Systems
SIGIR '24: Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information RetrievalPages 841–851https://doi.org/10.1145/3626772.3657893The rise of conversational recommender systems (CRSs) brings the evolution of the recommendation paradigm, which enables users to interact with the system and achieve dynamic recommendations. As one essential branch, multi-turn CRSs, built on the user ...
- research-articleJuly 2024
Resources for Brewing BEIR: Reproducible Reference Models and Statistical Analyses
SIGIR '24: Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information RetrievalPages 1431–1440https://doi.org/10.1145/3626772.3657862BEIR is a benchmark dataset originally designed for zero-shot evaluation of retrieval models across 18 different domain/task combinations. In recent years, we have witnessed the growing popularity of models based on representation learning, which ...
- research-articleJuly 2024
A Reproducibility Study of PLAID
SIGIR '24: Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information RetrievalPages 1411–1419https://doi.org/10.1145/3626772.3657856The PLAID (Performance-optimized Late Interaction Driver) algorithm for ColBERTv2 uses clustered term representations to retrieve and progressively prune documents for final (exact) document scoring. In this paper, we reproduce and fill in missing gaps ...
- research-articleJuly 2024
Revisiting Document Expansion and Filtering for Effective First-Stage Retrieval
SIGIR '24: Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information RetrievalPages 186–196https://doi.org/10.1145/3626772.3657850Document expansion is a technique that aims to reduce the likelihood of term mismatch by augmenting documents with related terms or queries. Doc2Query minus minus (Doc2Query-) represents an extension to the expansion process that uses a neural model to ...
- abstractJuly 2024
Improving Dropout Prediction for Informatics Bachelor Students
ITiCSE 2024: Proceedings of the 2024 on Innovation and Technology in Computer Science Education V. 2Pages 830–831https://doi.org/10.1145/3649405.3659472Despite numerous efforts to predict student dropout in university students, the exact variables influencing dropout remain elusive, as existing studies do not generalise well to other samples. The proposed PhD project uses meta science tools to ...
- research-articleSeptember 2024
Laying Foundations to Quantify the "Effort of Reproducibility"
JCDL '23: Proceedings of the 2023 ACM/IEEE Joint Conference on Digital LibrariesPages 56–60https://doi.org/10.1109/JCDL57899.2023.00018Why are some research studies easy to reproduce while others are difficult? Casting doubt on the accuracy of scientific work is not fruitful, especially when an individual researcher cannot reproduce the claims made in the paper. There could be many ...
- short-paperJuly 2024
Toward Evaluating the Reproducibility of Information Retrieval Systems with Simulated Users
ACM REP '24: Proceedings of the 2nd ACM Conference on Reproducibility and ReplicabilityPages 25–29https://doi.org/10.1145/3641525.3663619Reproducibility is a fundamental part of scientific progress. Compared to other scientific fields, computational sciences are privileged as experimental setups can be preserved with ease, and regression experiments allow the validation of computational ...
- ArticleJune 2024
Geospatial Webservices and Reproducibility of Research: Challenges and Needs
- Maxime Collombin,
- Massimiliano Cannata,
- Olivier Ertz,
- Gregory Giuliani,
- Jens Ingensand,
- Claudio Primerano,
- Daniele Strigaro
Web and Wireless Geographical Information SystemsPages 86–92https://doi.org/10.1007/978-3-031-60796-7_6AbstractThis article investigates challenges and requirements related to the reproducibility of geospatial research using geospatial web-services. Several researchers have identified hinders related to technology on the one hand, as well as challenges ...
- research-articleJune 2024
Croissant: A Metadata Format for ML-Ready Datasets
- Mubashara Akhtar,
- Omar Benjelloun,
- Costanza Conforti,
- Pieter Gijsbers,
- Joan Giner-Miguelez,
- Nitisha Jain,
- Michael Kuchnik,
- Quentin Lhoest,
- Pierre Marcenac,
- Manil Maskey,
- Peter Mattson,
- Luis Oala,
- Pierre Ruyssen,
- Rajat Shinde,
- Elena Simperl,
- Goeffry Thomas,
- Slava Tykhonov,
- Joaquin Vanschoren,
- Jos van der Velde,
- Steffen Vogler,
- Carole-Jean Wu
DEEM '24: Proceedings of the Eighth Workshop on Data Management for End-to-End Machine LearningPages 1–6https://doi.org/10.1145/3650203.3663326Data is a critical resource for Machine Learning (ML), yet working with data remains a key friction point. This paper introduces Croissant, a metadata format for datasets that simplifies how data is used by ML tools and frameworks. Croissant makes ...
- abstractJune 2024Best PosterBest Student Paper
Reporting Eye-Tracking Data Quality: Towards a New Standard
ETRA '24: Proceedings of the 2024 Symposium on Eye Tracking Research and ApplicationsArticle No.: 47, Pages 1–3https://doi.org/10.1145/3649902.3655658Eye-tracking datasets are often shared in the format used by their creators for their original analyses, usually resulting in the exclusion of data considered irrelevant to the primary purpose. In order to increase re-usability of existing eye-tracking ...
- research-articleJune 2024
Lazy Data Practices Harm Fairness Research
FAccT '24: Proceedings of the 2024 ACM Conference on Fairness, Accountability, and TransparencyPages 642–659https://doi.org/10.1145/3630106.3658931Data practices shape research and practice on fairness in machine learning (fair ML). Critical data studies offer important reflections and critiques for the responsible advancement of the field by highlighting shortcomings and proposing recommendations ...
- research-articleJune 2024
Experimental evaluation of a machine learning approach to improve the reproducibility of network simulations
A stochastic network simulation is verified when its distribution of outputs is aligned with the ground truth, while tolerating deviations due to variability in real-world measurements and the randomness of a stochastic simulation. However, comparing ...