|
Australasian Journal of Educational Technology, 2022, 38(3)Critical appraisal criteria[8] Peters et al 38-3Critical appraisal criteria
To critically appraise the included reviews and assess if they were conducted and reported according to
high-quality standards (Pollock et al., 2021), the Joanna Briggs Institute (JBI, (2017) checklist for
systematic reviews and research syntheses was used (see Appendix B). A scoring system consisting of
assigning scores to each review based on each of the 10 checklist questions was used. When the specific
criteria under consideration were explicitly and clearly reported in the review, one point was assigned;
when the review only partly or ambiguously described the criteria, a half point was assigned; and when the
criteria were missing, zero points were assigned. The scores were then summed to yield the final quality
score out of 10. To be included in the final synthesis, reviews had to meet a quality threshold of 5. The
results of the quality assessment were used to contextualise the overview’s evidence base and assess how
the systematic review methods may have affected the overview’s overall results (Pollock et al., 2021),
including implications for practice and research (Aromataris et al., 2015). After critical appraisal, 10 studies
were excluded, leaving 13 studies for evidence synthesis.
Data collection and synthesis
EPPI Reviewer software and an Excel worksheet were used to extract and code the data from the included
reviews. Data collection and synthesis were carried out in three phases. In the first phase, a data extraction
form was developed in an Excel worksheet as a logical approach for storing extracted information (i.e.,
review characteristics, purpose, synthesis methods and findings). Two reviewers who had been involved in
the screening and eligibility phases independently extracted the information from a random sample of 20%
of the included reviews. As in the screening phase, disagreements between the reviewers on the application
and interpretation of the extraction form were resolved through consensus. After reaching agreement, data
extraction was conducted on the remaining reviews by two independent reviewers. Using guidelines by
Aromataris et al. (2015), the following aspects of each publication were extracted: publication metadata
(i.e., publication year and type), review purpose, type of review, setting and context, number of databases
used, date range of included studies, number and types of studies included and country of origin of each
review, method for evidence synthesis, reported findings, implications for practice and future lines of
research.
In the second phase, after the extraction was completed, a hybrid coding scheme was developed, which
allowed for both, deductive (closed) and inductive (open) coding to occur in relation to the specific
objectives of the study in a flexible and emergent manner. This coding phase, which was implemented by
two independent reviewers, emphasised double-checking for accuracy, reliability and consistency of the
hybrid code scheme. In the third and final phase, qualitative content analysis (Schreier, 2012) was used to
generate literature summary tables of the included reviews and identify patterns in the coded data.
Australasian Journal of Educational Technology, 2022, 38(3).
126
Descriptive statistics were used to calculate the percentage of study characteristics reported across different
methodological aspects and systematically describe and synthesise the range of variables studied.
Dostları ilə paylaş: |
|
|