One of the key pillars of science is peer-review - scientific contributions are scrutinized by peer researchers. As a researcher, our primary aim is to publish our findings and thus to master the competition in the peer-review process.

Unfortunately, peer-review is not what it used to be: the number of papers submitted to conferences and journals is constantly growing and peer reviewers are facing increasing fatigue. A reason here is also, that it is increasingly difficult for peer-reviewers to identify the merits of a particular submission and credibly assess the progress beyond the state-of-the-art.

With the Open Research Knowledge Graph, we aim to address these issues and improve the lives of both paper authors and peer reviewers (in addition to the numerous other benefits ;-) )

• peer-reviewers are provided with a characterization of a submission by juxtaposing it with related approaches served on the silver tablet as an ORKG comparison
• authors have a tool for systematically describing and pre-assessing their research contribution, which will, once made available to the peer-reviewers, highlight the merits of the submission

What is an ORKG Comparison?

Comparisons organize contributions towards specific research problems in a tabular view and summarize the state-of-the-art in their corresponding research field. If applicable, data from comparisons can also be visualized in different ways, e.g. bar charts or scatter plots. Comparisons can be exported in various formats including LaTeX (and thus also directly included in the related work section of your article). ORKG comparisons receive a DOI and are citable as scholarly contributions in their own light.

How to create an ORKG comparison

You can get started contributing to the OKRG with the contribution editor - here two screencasts, explaining its use:

• illustrating the contribution editor with a computer science example

• how to describe a performance evaluation using a benchmark

You can alternatively follow these instructions:

1. Sign in.
2. Go to View -> Comparisons and click the "+Create comparison"-button to start your work.
3. You need to add research contributions you want to compare. If you already have a list of contributions, you can use the csv import (Option 1). If this is your first time working with the ORKG, we highly recommend choosing Option 2. To add new papers, click "Add contribution" and enter the DOI or title. If the contribution you want to add is already described in the ORKG, this is straightforward. If not, use the “Add new paper” button to create a new semantic description. The wizard will guide you through the process.
4. Repeat the "Add contribution" step for every contribution you want to include.
5. Once you are finished, click "Make Comparison" to get the resulting table. You can now customize your comparison, e.g. by reordering the properties in the table in a logical order. You can find this option, along with several others, under "More".
6. Finally, you can publish your comparison under More -> Publish. Provide metadata like title and research field and choose wether you wish to receive a doi for your comparison or not. Once you click the "Publish" button, a persistent link to your comparison is created. Provide this link in the submission form.

How should I structure my comparison?

For creating your comparison:

• think about what are the common and distinguishing features of your approach compared to related approaches - typically each of these features can be translated into one or multiple properties on the contribution description
• typically a contribution description should contain the information a well-crafted abstract would contain, i.e. the research problem addressed, details on the approach, the implementation, and evaluation
• add information facilitating the reproducibility of your approach, e.g. a link to a software repository, datasets, etc.

FAQ

How are double-blind submissions handled?
While creating your comparison you will be able to associate it with a conference. For conferences requiring double-blind submissions, we will then anonymize your submissions till the end of the review period. Please contact us if your conference is not yet listed.

What happens if my paper gets accepted or rejected
We will track accepted papers of listed conferences and associate them after acceptance with the proper DOIs of the published articles. If your submission is rejected, it is still worth keeping your comparison in the ORKG. You can update it for resubmission to another conference and/or link your publication to a preprint in an open-access repository like arXiv.

Should I cite my ORKG comparison from my submitted paper
Indeed citing your ORKG comparison from your paper seems to be a good way to make reviewers aware of how your work goes beyond the state-of-the-art in the field. You can directly add a link to the ORKG comparison and additionally add a citation via the assigned DOI.