Call for Papers

We invite contributions to BELIV 2018, the international forum to broadly discuss research methods in visualization. Our discussions span from novel and not-yet fully established evaluation methods for visualization tools and techniques to methods that more generally establish the validity and scope of acquired visualization knowledge. The 2018 call for papers includes an expanded scope from years past to reflect this newly broadened purview.

New this year: Complementary to the broader scope on research methods more generally, 2018 will also include a focus topic—replication practices in visualization—which will shape half of the workshop. This call for papers thus also includes specific issues related to both the replication of quantitative and qualitative work, as well as opportunities for reflective assessment of unsuccessful replication efforts.

Topics of Interest

The focus topic for BELIV 2018 is replication. We follow Peng et al.'s definition of replication [1] as: ``...the replication of scientific findings using independent investigators, methods, data" vs. reproducibility ``which requires that data sets and computer code be made available to others for verifying published results and conducting alternative analyses."[1]

Based on the definition of replication above, specific focus topics of interest include, but are not limited to:

  • Best practices for supporting replication
  • How do we, or how should we build on other’s work
  • The value of replication in VIS
  • Alternatives when replication is not possible
  • Discussion of methods that support replicability in VIS, e.g. pre-registration
  • Reflection on replication attempts, successes, and failures
  • Dangers of replication
  • Reviewing replicated work
  • Meta-analysis of replication in visualization

BELIV 2018 also seeks papers on the broader topics of interest of the workshop. Broad topics of interest include, but are not limited to:

  • Evaluation in the visualization development lifecycle
  • Novel evaluation methods
  • Revisiting or extending existing research methods
  • Synthetic data sets and benchmarks
  • Taxonomy of tasks
  • Benchmark development and repositories
  • How to turn empirical research into heuristics or guidelines
  • Reflections on evaluation practice in visualization in the past, present, and future
  • NEW: Validity of studies and research results
  • NEW: Evaluation criteria and methods for the paper reviewing process
  • NEW: Design methods and methodologies
  • NEW: Methodological frameworks and theories
  • NEW: Rigor in the evaluation process
  • NEW: How to best communicate the results of empirical research

Paper Types

The workshop focuses on contributions from all areas of visualization, including Visual Analytics, Information Visualization, and Scientific Visualization. We accept 3 types of submissions—research, position, and survey papers:

Research papers aim to make progress toward novel research methods in visualization and to discuss their benefits and limitations compared to traditional methodologies. As in past editions of BELIV, we will not accept papers that merely report on the use of a method or methodology unless the focus of the contribution is on lessons learned about applying a novel method and new insights on the method itself are presented. Research papers present new work and unpublished results on the topic areas of the workshop, including the focus topic. Research papers will be selected according to their novelty, quality, and relevance.

Position papers are problem discussions or statements describing the author's relevant experience and ideas in regards to methods and methodologies for visualization research, and in particular the focus topic of the workshop. Position papers will be selected according to their importance and relevance for the workshop topics and how well they will fit the planned discussions. To avoid confusion, please clearly label a position paper as such with a subtitle (e.g., by doing the following in Latex: \title{ Your Title \\ \vspace{1ex} \large Position Paper})

Survey papers are intended to provide up-to-date and comprehensive surveys on topics relevant to methods and methodologies for visualization research. We welcome submissions that introduce emerging methodological approaches, as well as proposals on traditional research methods. We also welcome contributions from related disciplines and other application areas, including, but not limited to human-computer interaction, psychology, statistics, social science, computational sciences, etc. All survey papers, however, must make a clear case for their (potential) relevance to visualization.

Format

Papers can include up to eight pages of content with additional pages for references only. The length of the submission should be commensurate with the contribution.

All submissions should be formatted in the VGTC conference paper style. Suitable templates, in LaTeX and Word, can be downloaded from here. The submission, however, must be made in PDF format. Authors can decide whether they want to reveal their names on the submission (single-blind) or submit it anonymously (double-blind).

Submission

To submit a paper create an account and submit the paper to the submission system at: Precision Conference ("Society: VGTC", "Conference/Journal: VIS 2018", "Track: VIS 2018 BELIV Workshop"). Please clarify whether you are submitting a position, research, or survey paper.

Publication

Accepted papers will be published in the IEEE digital library, including the assignment of DOIs to individual papers. BELIV papers are thus considered to be archival and can be referenced. While BELIV papers that are extended with at least 30% new content may be re-used for publication in a journal (subject to the rules of that journal), this content cannot be re-used for a future VIS, EuroVis, or PacificVis conference submission.

[1] Peng, R. (2009). Reproducible research and Biostatistics Biostatistics, 10 (3): 405-408 .