+
+
+
+
0
0
+
+
+
+
9
Sánchez-Caballé et al. (2020)
+
+
+
+
0
0
+
+
+
0
8.5
Zhao et al. (2021)
+
+
+
+
+
-
-
+
+
+
8
Fernández-Batanero et al.
(2020)
+
+
+
+
-
-
0
+
+
+
7.5
Starkey (2020)
+
+
+
+
-
-
+
+
+
0
7.5
Esteve-Mon et al. (2020)
+
+
+
+
-
-
-
+
+
+
7
Perdomo (2020)
-
+
+
+
0
-
+
+
0
+
7
Palacios et al. (2020)
-
+
+
+
0
-
+
+
+
-
6.5
Pettersson (2018)
-
+
+
+
-
-
-
+
+
+
6
Duran et al. (2016)
-
0
+
+
-
-
-
+
0
+
5
Recio et al. (2020)
-
+
+
+
-
-
-
+
0
0
5
Rodriguez-García et al. (2019)
+
+
+
+
-
-
-
+
-
-
5
Note . 1. Presence of a review question; 2. Use of inclusion criteria; 3. Use of search strategy; 4. Sources
and resources used to search were reported; 5. Criteria for study appraisal; 6. Critical appraisal conducted
by two or more reviewers; 7. Minimise errors in data extraction; 8. Methods used to combine studies; 9.
Recommendations for policy and/or practice; 10. Implications for future research; + = Clearly reported; 0
= Partially reported; - = Not reported.
Surprisingly, just over half of the reviews (61%) included a guiding review question, an essential step that
helps define the scope when conducting systematic reviews and establish effective search strategies. Most
reviews clearly reported inclusion criteria, search strategies and sources and resources used for the review
process. However, critical appraisal was another surprising method that was absent from most reviews
(85%). Only two reviews assessed the quality of included primary studies and no reviews explicitly reported
critical appraisal being conducted by two or more reviewers independently. This finding has implications
as there is no indication of the quality of the included primary studies in most reviews, bringing into
question the reliability and validity of the overall findings. Data extraction was another clear limitation, as
under half of the reviews (38%) reported methods to minimise errors in this critical phase. Another area of
concern were the recommendations for policy and practice and specific directives for future lines of
research. In both criteria, there should be a clear link to the results of the review. In the lower-quality cluster,
three reviews had vague recommendations for policy or practice and one review did not report any.
Concerning future lines of inquiry, three reviews had ambiguous implications for future research, which
Australasian Journal of Educational Technology, 2022, 38(3). 131
did not clearly relate to the results of the review, while two reviews did not report any gaps in the research
requiring future attention.