SUPPORT Summaries
Evidence of the effects of health system interventions
for low- and middle-income countries
Evidence of the effects of health system interventions
for low- and middle-income countries
August, 2016 - SUPPORT Summary of a systematic review | print this article | download PDF
Internet based learning is increasingly used in the training and ongoing education of health professionals. Concerns about the effects of this medium of teaching and learning have stimulated a large body of research. This summary describes a review and meta-analysis of research on the effectiveness of internet based learning by health professions.
Key messages
-Practise exercises, tutorials, online discussions and longer duration courses may improve the effects of internet-based learning.
People who make decisions about use of the internet for learning in health professions.
Cook DA, Levinson AJ, Garside S, et al. Internet-based learning in the health professions: a meta-analysis. JAMA 2008; 300:1181-96.
A summary of studies addressing a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise the relevant research, and to collect and analyse data from the included studies.
SUPPORT – was an international project to support the use of policy relevant reviews and trials to inform decisions about maternal and child health in low- and middle-income countries, funded by the European Commission (FP6) and the Canadian Institutes of Health Research.
Internet-based learning has become a popular approach to medical education since the advent of the World Wide Web in 1991. It permits learners to participate at a time and place convenient to them, facilitates innovation in instructional methods, and potentially allows instruction to be tailored to the individuals needs.
Several studies have been undertaken to assess the effectiveness of internet–based learning. This review included sutdies of the use of internet-based learning at any stage in training or practice by health professionals. The authors also sought to determine factors that could explain differences in effect across participants, settings, interventions, outcomes and study designs.
| Review objectives: To assess the effects of internet-based learning by health professionals | ||
| Type of | What the review authors searched for | What the review authors found |
|---|---|---|
| Study designs & interventions | Internet based learning for health professionals at any stage of training or practice |
201 studies (including observational and experimental designs) of internet based learning for health professionals, addressing a wide range of topics, and using a range of modalities for teaching and learning |
| Participants | Health profession learners (including students and practising physicians, nurses, dentists, pharmacists and others) | Health profession learners |
| Settings | All settings and languages |
All settings |
| Outcomes | Satisfaction; learning (knowledge, attitudes, skills); behaviours or effects on patients | Knowledge, skills, behaviours and effects on patients, satisfaction |
| Date of most recent search: January 2008 | ||
| Limitations: The review is from 2008 and the studies up to 2007. New information is likely to be available. | ||
To assess the effects of internet-based learning by health professionals
Cook DA, Levinson AJ, Garside S, et al. Internet-based learning in the health professions: a meta-analysis. JAMA 2008; 300:1181-96.
High: This research provides a very good indication of the likely effect. The likelihood that the effect will be substantially different† is low.
Moderate: This research provides a good indication of the likely effect. The likelihood that effect will be substantially different† is moderate.
Low: This research provides some indication of the likely effect. Howevver, the likelihood that it will be substantially different† is high.
Very low: This research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different† is very high.
* This is sometimes referred to as 'quality of evidence' or 'confidence in the estimate'.
† Substantially different = a large enough difference that it might affect a decision.
See last page for more information.
Findings are reported separately for the two main comparisons - internet-based learning compared to no intervention and compared to non-internet-based learning. Key findings of sub-analyses are also reported.
Internet based learning was compared to no learning intervention for health professionals. Satisfaction outcomes were not reported, as no studies reported meaningful outcomes of this type.
Internet-based learning compared to no intervention |
||||
| People: Health professionals Settings: Classroom or practice Intervention: Internet-based learning Comparison: No intervention |
||||
| Outcomes | Impact | Certainty of the evidence (GRADE) |
||
|
Knowledge |
There was a large effect on knowledge outcomes. Pooled effect size 1.00 (95% CI 0.90 to 1.10) High interactivity, ongoing access to course materials, online discussion, not associated with larger effects. High quality studies were associated with smaller effects. Pooled effect size 0.71 (95% CI 0.51 to 0.92) |
Low |
||
|
Skills |
Overall, the effect on skills was uncertain. Pooled effect size 0.85 (95% CI 0.49 to 1.20). Practice exercises were associated with larger effects. Pooled effect size 1.01 (95% CI 0.60-1.43) High interactivity, repetition, and online discussion were not associated with larger effects. |
Very Low |
||
|
Behaviours and patient care |
Overall, the effects on behaviours and patient care were uncertain. Pooled effect size 0.82 (95% CI 0.63 to 1.20) Tutorials, longer duration courses and online peer discussion were associ-ated with larger effects. |
Very Low
|
||
GRADE: GRADE Working Group grades of evidence (see above and last page). CI = confidence interval.
2) Internet-based learning compared to non-internet-based learning
Internet-based learning by health professionals was compared to alternative instructional media, including face-to-face teaching and learning.
Internet-based learning compared to non-internet-based learning |
||||
|
People: Health professionals Settings: Classroom or practice Intervention: Internet-based learning Comparison: Non-internet-based learning interventions |
||||
| Outcomes |
Impact Standardised mean differences |
Certainty of the evidence (GRADE) |
||
|
Knowledge |
Overall, the effect on knowledge was uncertain. Pooled effect size 0.12 (95% CI 0.003 to 0.24) Internet courses using online discussion and longer courses were associated with larger effects. |
Very Low |
||
|
Skills |
Overall, the effect on skills was uncertain. Pooled effect size 0.09 (95% CI -0.26 to 0.44) High levels of interactivity, practice exercises and peer discussion were associated with larger effects. |
Very Low |
||
|
Behaviours and patient care |
Overall, the effect on behaviours and patient care was uncertain. Pooled effect size 0.51 (95% CI -0.24 to 1.25) Online discussion and single instance interventions were associated with larger effects. |
Very Low |
||
|
Satisfaction |
Overall, the effect on satisfaction was uncertain. Pooled effect size 0.10 (95% CI -0.12 to 0.32) Short courses and single instance rather than ongoing access internet-based interventions were associated with larger effects. |
Very Low |
||
|
GRADE: GRADE Working Group grades of evidence (see above and last page). CI = confidence interval. |
||||
| Findings | Interpretation* |
|---|---|
| APPLICABILITY | |
|
Studies in all languages were included, but authors did not indicate whether this included studies from low- income countries. |
|
| EQUITY | |
|
Impacts on equity were not reported. |
|
| ECONOMIC CONSIDERATIONS | |
|
No studies reported costs or cost-effectiveness. |
|
| MONITORING & EVALUATION | |
|
|
|
*Judgements made by the authors of this summary, not necessarily those of the review authors, based on the findings of the review and consultation with researchers and policymakers in low-income countries. For additional details about how these judgements were made see: www.supportsummaries.org/methods |
|
Related literature
Cook DA, Levinson AJ, Garside S, et al. Instructional design variations in internet-based learning for health profes-sions education: a systematic review and meta-analysis. Acad Med 2010; 85:909-22.
Ruiz JG, Mintzer MJ, Leipzig RM. The Impact of e-learning in medical education. Acad Med 2006; 81:207-12.
This summary was prepared by
Gabriel Rada, Pontificia Universidad Católica de Chile; and Lilian D. Dudley, Faculty of Health Sciences, University of Stellenbosch, South Africa.
Conflict of interest
None declared. For details, see: www.supportsummaries.org/coi
Acknowledgements
This summary has been peer reviewed by: Fernando Althabe and David Cook.
This review should be cited as
Cook DA, Levinson AJ, Garside S, et al. Internet-based learning in the health professions: a meta-analysis. JAMA 2008; 300:1181-96.
The summary should be cited as
Gabriel Rada, Lilian Dudley. Does Internet based learning in the health profes-sions improve learning outcomes? A SUPPORT Summary of a systematic review. August 2016. www.supportsummaries.org
Keywords
internet-based learning, distance learning, distance education, evidence-informed health policy, evi-dence-based, systematic review, health systems research, health care, low and middle-income countries, developing countries, primary health care.
