Call for Papers

We invite contributions to BELIV 2016, the international workshop on evaluation in visualization.

Established scientific methods like controlled experiments and standard metrics like time and error continue to be the workhorse of evaluation in visualization.Yet, there is a growing need for the visualization community to develop dedicated evaluation approaches and metrics for evaluation at all stages of the development life cycle that address specific needs in visualization. The goal of the workshop is to continue the discussion and spread the word on alternative and novel evaluation methods and methodologies for visualization.

Paper Types

The workshop focuses on contributions from all areas of visualization, including Visual Analytics, Information Visualization, and Scientific Visualization. We accept 3 types of submissions—research, position, and survey papers:

Research papers: aim to make progress toward novel evaluation methods in visualization and to discuss their benefits and limitations compared to traditional methodologies. As in past editions of BELIV, we will not accept papers that merely report on the evaluation of a system or technique unless the focus of the contribution is on lessons learned about applying a novel evaluation method or methodology and new insights on the evaluation itself are presented. Research papers present new work and unpublished results on the topic areas of the workshop. Research papers will be selected according to their novelty, quality and relevance.

Position papers are problem discussions or statements describing the author's relevant experience and ideas in regards to evaluation methods and methodologies for visualization. Position papers will be selected according to their importance and relevance for the workshop topics and how well they will fit the planned discussions.

Survey papers are intended to provide up-to-date and comprehensive surveys on topics relevant to the evaluation of visualization. We welcome submissions that introduce emerging methodological approaches, as well as proposals on traditional evaluation topics. We also welcome contributions from related disciplines and other application areas, including, but not limited to human-computer interaction, psychology, statistics, social science, computational sciences, etc. All survey papers, however, must make a clear case for their (potential) relevance to visualization.

All types of papers can be up to eight pages of text long but the length of a submission needs to correspond to its contribution. Additional pages for references are allowed, but any extra page can only contain references and nothing else.

All papers will be peer-reviewed by members of the program committee as well as the organizers.

Publication

As for previous BELIVs, we are working on making sure that we will be able to publish all accepted papers in the ACM digital library including the assignment of DOIs to the papers.

Submission

To submit a paper create an account and submit the paper to the submission system at: https://cmt3.research.microsoft.com/BELIV2016/. Please clarify whether you are submitting a position, research, or survey paper.

Format

All submissions should be formatted in the ACM style. Suitable templates, in LaTeX and Word, can be downloaded from: https://www.acm.org/publications/proceedings-template/. The submission, however, must be made in PDF format. Authors can decide whether they want to reveal their names on the submission (single-blind) or submit it anonymously (double-blind).

Topics of Interest

Topics of interest include, but are not limited to:

  • Evaluation in the visualization development lifecycle,
  • Utility characterization,
  • Evaluation metrics,
  • Insight characterization,
  • Synthetic data sets and benchmarks,
  • Taxonomy of tasks,
  • Computational evaluation,
  • Benchmark development and repositories,
  • Methods for longitudinal studies and adoption,
  • Evaluation of early prototypes,
  • Evaluation heuristics and guidelines, and
  • Reflections on evaluation practice in visualization in the past, present, and future.
Topics that are not of interest:
  • Evaluation results of a visualization system or technique

Important Dates

June 19, 2016: Paper submission due
July 10, 2016: First notification
July 20, 2016: Revisions due
Aug 5, 2016: Final notification
Aug 31, 2016: Camera-ready due
Oct 24, 2016: BELIV workshop (1-day)

Papers from previous BELIVs

For past BELIV workshops and papers see these links:
BELIV 2006: published in the ACM DL
BELIV 2008: published in the ACM DL
BELIV 2010: published in the ACM DL
BELIV 2012: published in the ACM DL
BELIV 2014: published in the ACM DL