BELIV 2024 Papers


Session 1: Welcome, Keynote and Q&A, and Paper Talks
(8:30am - 9:45am EST)

8:30 - 8:40: Welcome Opening

8:40 - 9:25: Keynote
Dr. Simone Stumpf — How to Empower Stakeholders to Assess AI systems
Simone Stumpf

Assessing AI to make sure it is ethical and responsible has, until now, been left solely in the hands of AI experts. There is no agreement of what counts as ‘good enough’ to be acceptable to stakeholders, including ones who ultimately use the AI or are affected by its outputs. PHAWM, one of three RAI UK-funded keystone projects, is driving change in AI testing and evaluation through the novel concept of participatory AI auditing, where a diverse set of stakeholders without a technical background in AI, such as domain experts, regulators, decision subjects and end-users, are empowered to undertake audits of predictive and generative AI, either individually or collectively. This talk will describe the landscape of AI assessment, particularly from a stakeholder perspective. [More Info]


9:35 - 9:45: Paper Visualising Lived Experience: Learning from a Master Narrative Framing
Mai Elshehaly, Mirela Reljan-Delaney, Jason Dykes, Aidan Slingsby, Jo Wood, Sam Spiegel


BREAK 9:45 - 10:15


Session 2: Experiments and Evaluations (Paper Presentations) + Break-out Session (10:15am - 11:30am EST)


10:15 - 10:20: Session Intro



10:40 - 10:50: Paper Tasks and Telephones: Threats to Experimental Validity due to Misunderstandings of Visualisation Tasks and Strategies
Abhraneel Sarma, Sheng Long, Michael Correll, Matthew Kay

10:50 - 11:00: Paper "Normalized Stress" is Not Normalized: How to Interpret Stress Correctly
Kiran Smelser, Jacob Miller, Stephen Kobourov

11:00 - 11:30 Breakout Session #1


BREAK 11:30-12:00


Session 3: Theory and Practice (Paper Presentations) (12:00pm - 1:15pm EST)


12:00 - 12:05: Session Intro

12:05 - 12:15: Paper The Role of Metacognition in Understanding Deceptive Bar Charts
Antonia Schlieder, Jan Rummel, Peter Albers, Filip Sadlo


12:25 - 12:35: Paper Design-Specific Transforms In Visualization
Eugene Wu, Remco Chang

12:35 - 12:45: Paper Complexity as Design Material
Florian Windhager, Alfie Abdul-Rahman, Mark-Jan Bludau, Nicole Hengesbach, Houda Lamqaddam, Isabel Meirelles, Bettina Speckmann, Michael Correll

12:45 - 12:55: Paper Visualization Artifacts are Boundary Objects
Jasmine Tan Otto, Scott Davidoff

12:55 - 1:05: Paper Merits and Limits of Preregistration for Visualization Research
Lonni Besançon, Brian Nosek, Tamarinde Haven, Miriah Meyer, Cody Dunne, Mohammad Ghoniem



BREAK 1:15 - 1:45pm


Session 4: Reflections and Research Directions (Paper Presentations) + Break-out Session (1:45pm - 3:00pm EST)


1:45 - 1:50: Session Intro

1:50 - 2:00: Paper Testing the Test: Observations When Assessing Visualization Literacy of Domain Experts
Seyda Öney, Moataz Abdelaal, Kuno Kurzhals, Paul Betz, Cordula Kropp, Daniel Weiskopf

2:00 - 2:10: Paper We Don't Know How to Assess LLM Contributions in VIS/HCI
Anamaria Crisan


2:20 - 2:30: Paper Striking the Right Balance: Systematic Assessment of Evaluation Method Distribution Across Contribution Types
Feng Lin, Arran Zeyu Wang, Md Dilshadur Rahman, Danielle Albers Szafir, Ghulam Jilani Quadri

2:30 - 2:55 Breakout Session #2

2:55 - 3:00 Closing