Figure 6.1. The procedure for verifying eligibility for systematic checks or systematic cards A new screening can be carried out during the accession negotiations, if EU legislation has been updated. Cohens Kappa (Altman, 1991) is a widespread statistic for assessing screener compliance, which takes into account the degree of concordance between screens that would occur randomly. However, the interpretation of Kappa scores is subjective, as there is no consensus on scores that indicate an „appropriate” match and the very concept of „appropriate” concordance is subjective. An adequate allocation of the verification team for the exercise of the aptitude examination with regard to the number and expertise of the parties involved is important to ensure effectiveness (DEFRA, 2015) and can help minimise the risk of errors or distortions. If the members of the review team are authors of articles identified in the searches, the assignment of screening tasks should ensure that the members of the review team have no influence on decisions about the suitability of their own articles. It is important that all adequacy verification decisions are recorded so that judgments made during the systematic verification or systematic map are transparent and, where appropriate, acceptable (e.g. .B. when readers ask why a given study was not included). A record of screening decisions (e.g.
B in a literature management tool or relational database) which can be easily accessed to display articles containing, excluded or deemed ambiguous at each selection stage. The tool or database containing the complete set of screening decisions should be archived so that it can be made available to readers upon request for the systematic verification or systematic map report. Eligibility decisions include an assessment and errors or biases may occur during the aptitude examination if the process is not conducted carefully. Any limitations in the suitability screening process should be mentioned in the Discussion (or Critical Thinking) section of the final evidence synthesis report, so that readers can take them into account when interpreting the overall results of the evidence synthesis (Liberati et al., 2009). . . .