Call for Papers

We invite contributions to BELIV 2022, the international forum to broadly discuss research methods in visualization. Our discussions span from novel and not-yet fully established design and evaluation methods for visualization tools and techniques, to methods that more generally establish the validity and scope of acquired visualization knowledge. BELIV supports contributions and discussions from the rich spectrum of visualization researchers, embracing the varied ways that researchers self-identify with respect to the main conference areas at VIS.

This year: Complementary to the broad scope of BELIV and its general commitment to research methods, BELIV 2022 will include a timely focus topic: Designing and Evaluating Visualizations for an Ethical, Inclusive, and Responsible Future.

This topic is focused on considering how our visualization practices and processes can shape the future for the better or for the worse. As with all innovations, visualization and visualization research do not occur in a vacuum. Rather, as a community, we must consider how we can put ethics in the forefront of our research agendas and promote inclusivity in our research practices. We need to consider the contexts and challenges that lead to our visualization work, the means by which that work is realized, and how it can be leveraged for disparate purposes beyond what its original stakeholders envisioned. Ensuring that we contribute to an ethical, inclusive and responsible future means not just considering the grand challenges and directions of our community, but also how we as individual researchers shape the future for individuals, our communities, and societies. This line of thought is especially critical in light of how the COVID-19 pandemic has reshaped (and has the potential to further reshape) how we perform our research, reflect on our impact, and conceive of our field. Many people are asking:what will be the new normal?

We are interested in the practices and methods that visualization practitioners are deploying—and the discussions that they are having—to ensure that visualization contributes to the realization of a better future. To this end, we are soliciting papers that describe experiences and challenges in conducting visualization research and evaluations in ways that are ethical, inclusive, and responsible. Recommended paper topics include:

Topics of Interest

Reflections on inclusivity and population representation in visualization methods, studies, and participation

  • Expanding the scope of visualization research to less-typically-included populations, such as non-WEIRD (Western, industrialized, rich, and democratic) populations, marginalized populations, populations with different accessibility needs, and sensitive populations that might be at risk (kids, adolescents, others)
  • Exploring the differing impacts of visualization research and design on disparate populations, and the interplay of visualizations and existing structures of power or oppression
  • Methods for designing and evaluating visualizations from non-WEIRD cultural and societal perspectives
  • Strategies to promote inclusivity in design

Ethical considerations in visualization research

  • Approaches and frameworks for considering fairness and ethics when designing and evaluating visualizations
  • Applying ethical discussions and approaches from other fields to visualization
  • Approaches to elicit stakeholder perspectives and needs related to sensitive items
  • Facilitating discussions about how our tools could be used and their ethical implications

The responsible visualization practitioner

  • Approaches to identifying and mitigating personal biases during visualization design and evaluation
  • Assessing and addressing our prejudices and perceptions of stakeholders, especially end-users
  • The role of researchers' culture and background in intre/intra-cultural relationships
  • Evaluating potentially malicious use of visualization

BELIV 2022 also seeks papers on the broader topic of research methods in visualization, which include, but are not limited to:

Innovative Approaches to Visualization Evaluation

  • Novel evaluation methods
  • Revisiting or extending existing research methods
  • Data analysis techniques that are less used in the visualization community, such as Bayesian statistical analysis methods, multiverse analysis, and methods from vision science
  • Evaluation metrics for subjective decisions using visualization
  • Synthetic data sets and benchmarks
  • ML and visualization, how to evaluate appropriate trust (when to trust on not trust ML)

Innovative Approaches to Visualization Design

  • Novel design approaches for visualization research
  • Revisiting or extending existing design and evaluation methods of human-ML systems

Reflective Practices and Insights on Visualization

  • How to turn empirical research into heuristics or guidelines
  • Reflections on evaluation practice in visualization in the past, present, and future
  • Meta-analysis of existing work in visualization from a methods and methodology perspective
  • Taxonomies / ontologies of evaluation methods, highlighting underexplored or underexplored methods for visualization
  • Methodological frameworks and theories

Foundations of Visualization

  • Rigor in the evaluation process
  • Theoretical and epistemological considerations for visualization research
  • Validity of studies and research results
  • Alternatives when replication is not possible
  • What are the frontiers and limits in visualization research?
  • Effects of visualization on internal cognitive processes

Communicating and Reviewing Visualization Research

  • Evaluation criteria and methods for the paper reviewing process
  • How to best communicate the results of empirical research
  • Methods on documenting, archiving and replicating qualitative research

Visualization Evaluation & Design in Challenging Settings

  • Evaluation in the visualization development lifecycle
  • Longitudinal studies/long-term methodologies and challenges
  • Evaluation in complex confidential or sensitive settings (visualization in war rooms)
  • Design and evaluation methods in virtual and hybrid settings
  • Post-pandemic reflections on the future of visualization research

Paper Types

The workshop focuses on contributions from all areas of visualization, including VAST, InfoVis, and SciVis. We accept 3 types of submissions— research, position, and survey papers:

Research papers im to make progress toward novel research methods in visualization and to discuss their benefits and limitations compared to traditional methodologies. As in past editions of BELIV, we will not accept papers that merely report on the use of a method or methodology unless the focus of the contribution is on lessons learned about applying a novel method and new insights on the method itself are presented. Research papers present new work and unpublished results on the topic areas of the workshop, including the focus topic. Research papers will be selected according to their novelty, quality, and relevance.

Position papers are problem discussions or statements describing the author's relevant experience and ideas in regards to methods and methodologies for visualization research, and in particular the focus topic of the workshop. Position papers will be selected according to their importance and relevance for the workshop topics, and how well they will fit the planned discussions.

Survey papers are intended to provide up-to-date and comprehensive surveys on topics relevant to methods and methodologies for visualization research. We welcome submissions that introduce emerging methodological approaches, as well as proposals on traditional research methods. We also welcome contributions from related disciplines and other application areas, including, but not limited to human-computer interaction, design, psychology, statistics, social science, computational sciences, etc. All survey papers, however, must make a clear case for their (potential) relevance to visualization.

Format

Papers can include up to eight pages of content with additional pages for references only. Both full-length and shorter papers are welcome. The length of the submission should be commensurate with the contribution.

All submissions should be formatted in the VGTC conference paper style. Suitable templates, in LaTeX and Word, can be downloaded from: https://tc.computer.org/vgtc/publications/conference/. The submission, however, must be made in PDF format. Authors can decide whether they want to reveal their names on the submission (single-blind) or submit it anonymously (double-blind).

Submission

To submit a paper, create an account and submit the paper to the submission system at: https://new.precisionconference.com/user/login. Please clarify whether you are submitting a position, research, or survey paper.

Publication

We plan to publish accepted papers in the IEEE digital library, including the assignment of DOIs to individual papers.