by Vivian van der Werf and Shirley de Wit
In the last month, we, Vivian and Shirley, joined a double-blind reviewing process for a big computer science education conference - to which we also each submitted a paper ourselves. Reviewing for this conference is voluntary and for both of us one of the first times to experience the reviewing process “from the inside”.
After signing up as a reviewer, we were asked to bid on the submitted abstracts. Each reviewer needed to indicate at least 24 papers that they wanted or maybe wanted to review (choosing from almost 400 submissions!). About a week later we were assigned 3-4 papers. Our daily supervisor kindly helped us to start the process of reviewing and gave us tips on what to look for and how to formulate our opinions. This was very helpful as we started to look at papers from a different perspective - a perspective that resulted in some useful insights for when you are writing yourself. For instance, the readability of the paper (both the structure and grammar) is important and can distract reviewers from really diving into the (content) quality of the paper. We also noticed that some of the papers started very promising, but when reading them more closely, we started doubting the quality of some, leaving us to wonder if we were maybe too critical in our reviews. Luckily, there are other reviewers with whom we could share opinions in the discussion phase. Sometimes our views aligned but for other papers opinions were conflicting, after which interesting discussions followed. It also made us wonder about our submissions; what will reviewers have to say about them?
Overall, we both think the reviewing process has been valuable. We learned to read papers from a reviewer's perspective and also about the struggles you can face: e.g. really liking the topic of a paper while the work might not be grounded in the existing literature. This mindset will help us when writing our papers, knowing better now what reviewers might look for. Furthermore, reading other literature after the review process makes us realize what the paper went through and also that, even though a paper gets accepted, not all reviewers may have agreed. Lastly, it is nice to contribute to the community, after all, we want the community to also evaluate our work!
* For more info on “reviewer #2’: https://onlinelibrary.wiley.com/doi/10.1111/ssqu.12824 Note that it might actually be number 3 that is most critical…