The notifications for acceptances have been sent out on March 5th. Congratulations to all who will be joining us in August. Thank you to the Program Committee members who have taken the time and energy to make difficult decisions during what we are calling, the Secondary Review process.
We received a large number of proposals (543) of excellent quality. The acceptance scores are as follows:
– Long Papers: 94 of 152 submissions accepted (61%)
– Short Papers: 120 of 214 submissions accepted (56%)
– Posters: 77 of 112 submissions accepted (69%)
– Panels: 28 of 40 submissions accepted (70%)
– Workshops: 15 of 22 submissions accepted (68%)
The overall acceptance rate (across all submission formats) is 72%.
One of the most important tasks of the PC is to make a determination to accept or reject submissions where reviewers have differing opinions. This document outlines our approach to this important task and documents our processes.
In line with Alex Wermer-Colan’s work and Roopika Risam’s documentation for ACH Conference, we aimed to conduct manual qualitative analysis of each of the reviews in January-February before we come to a final list for accepted submissions. The review response phase is not in the timeline of events this time around, in favor of the qualitative analysis of reviews approach. We want to be mindful of the labor that reviewers are volunteering for the DH community, as we deliberate on the acceptances. The number of submissions’ reviews to be evaluated also was manageable since it is back to the scale of the pre-pandemic conference this year (https://dh2024.adho.org/dh2024-submissions/)
The intention with the Secondary review process was to exert qualitative as well as quantitative evaluation of the reviews. Conftool allows for sorting by the average of the points given for each submission, as well as the consensus span score that indicates the differing opinions between reviewers. While this was the jumping off point for the higher level view of the submissions, the PC was asked to go through a number of submissions by reviews each. Similar to the review process, we tried our best to match the multilingual secondary reviews for the submissions with PCs who were knowledgeable of the language and the subject matter as much as possible. Submissions which did not have abstracts, file uploads, or incomplete information were filtered out of this process. These were often in the lower bound (red) in the sorting. PC decisions were especially important for the submissions placed in the middle in the yellow. The steps included making note of the new review criteria, submission type, and making comments on the Conftool Online Forum for the Program Committee. We developed a vocabulary for evaluating reviews in Subject Header of Online Forum Board:
- AGREE WITH (AW)
- DISQUALIFIED (DQ)
- TIE-BREAKING (TB)
- When reviewer gives informed opinion and PC member agrees with that opinion: Agree with Thoughtful Reviewer identified by their Reviewer Number
- When a reviewer does not provide enough information to be a valid review: Disqualified review
- When a reviewer is overly critical to a disproportionate degree: Disqualified review
- When all reviewers’ reviews are disqualified: PC Reviews
- When 2 reviewers vs 1 reviewer: Tie-breaker
- When reviews are not clear in the content of their review: PC goes in reviews the submission itself & makes a decision
For example:
Dr. Andrew Janco has created a new message to contribution ###: "DH Submission ###" Created at: 30th Jan 2024, 08:40:52am Vote: Maybe reject AW 1,2; TB 29-70 Review 3 overly-generous, 4 critical, 1+2 note lack of specifics
—————-
Acceptance numbers
A note to those who received rejections: It is always unpleasant to receive notes on our work. We recognize that peer review is a hard process. We appreciate you sharing your work with the Digital Humanities community. While we’re unable to change decisions, we highly encourage you to submit to ACH and / or submit again in the future, making note of your specific considerations.