On the Meaning of Medical Evidence Hierarchies

Authors

  • Jesper Jerkert Department of Philosophy and History, Division of Philosophy, KTH Royal Institute of Technology

DOI:

https://doi.org/10.5195/pom.2021.31

Keywords:

evidence-based medicine, evidence hierarchy, strength of evidence, quality of evidence, order relation, evidence aggregation, lexicographic ordering, GRADE, ceteris paribus, RCT, observational study

Abstract

Evidence hierarchies are investigative strategies ordered with regard to the claimed strength of evidence. They have been used for a couple of decades in EBM, particularly in assessing evidence for treatment recommendations, but remain controversial. An under-investigated question is what the order in the hierarchy means. Four interpretations are discussed here. The two most credible are “typically stronger” or “ideally stronger.” The well-known GRADE framework seems to assume some “typically stronger” reading. Even if the interpretation of an evidence hierarchy were established, hierarchies are rather unhelpful for the task of evidence aggregation. Specifying the intended order relation may help to sort out disagreements.

References

Andrews, Jeff, Gordon Guyatt, Andrew D. Oxman, Phil Alderson, Philipp Dahm, Yngve Falck-Ytter, Mona Nasser, Joerg Meerpohl, Piet N. Post, Regina Kunz et al. 2013. “GRADE Guidelines: 14. Going from Evidence to Recommendations: The Significance and Presentation of Recommendations.” Journal of Clinical Epidemiology 66, no. 7: 719–725. https://doi.org/10.1016/j.jclinepi.2012.03.013.

Andrews, Jeff, Holger J. Schünemann, Andrew D. Oxman, Kevin Pottie, Joerg J. Meerpohl, Pablo Alonso Cello, David Rind, Victor M. Montori, Juan Pablo Brito, Susan Norris et al. 2013. “GRADE Guidelines: 15. Going from Evidence to Recommendation: Determinants of a Recommendation’s Direction and Strength.” Journal of Clinical Epidemiology 66, no. 7: 726–735. https://doi.org/10.1016/j.jclinepi.2013.02.003.

Balshem, Howard, Mark Helfand, Holger J. Schünemann, Andrew D. Oxman, Regina Kunz, Jan Brozek, Gunn E. Vist, Yngve Falck-Ytter, Joerg Meerpohl, Susan Norris et al. 2011. “GRADE Guidelines: 3. Rating the Quality of Evidence.” Journal of Clinical Epidemiology 64, no. 4: 401–406. https://doi.org/10.1016/j.jclinepi.2010.07.015.

Bluhm, Robyn. 2009. “Some Observations on ‘Observational’ Research.” Perspectives in Biology and Medicine 52, no. 2: 252–263. https://doi.org/10.1353/pbm.0.0076.

Broadbent, Alex. 2019. Philosophy of Medicine. Oxford: Oxford University Press.

Brunetti, Massimo, Ian Shemilt, Silvia Pregno, Luke Vale, Andrew D. Oxman, Joanne Lord, Jane Sisk, Francis Ruiz, Suzanne Hill, Gordon H. Guyatt et al. 2013. “GRADE Guidelines: 10. Considering Resource Use and Rating the Quality of Economic Evidence.” Journal of Clinical Epidemiology 66, no. 2: 140–150. https://doi.org/10.1016/j.jclinepi.2012.04.012.

Concato, John. 2004. “Observational versus Experimental Studies: What’s the Evidence for a Hierarchy?” NeuroRx 1, no. 3: 341–347. https://doi.org/10.1602/neurorx.1.3.341.

Evidence-Based Medicine Working Group. 1992. “Evidence-Based Medicine: A New Approach to Teaching the Practice of Medicine.” Journal of the American Medical Association 268, no. 17: 2420–2425. https://doi.org/10.1001/jama.1992.03490170092032.

Grossman, Jason and Fiona J. MacKenzie. 2005. “The Randomized Controlled Trial: Gold Standard, or Merely Standard?” Perspectives in Biology and Medicine 48, no. 4: 516–534. https://doi.org/10.1353/pbm.2005.0092.

Gugiu, P. Cristian and Mihaiela Ristei Gugiu. 2010. “A Critical Appraisal of Standard Guidelines for Grading Levels of Evidence.” Evaluation & the Health Professions 33, no. 3: 233–255. https://doi.org/10.1177%2F0163278710373980.

Guyatt, Gordon, Andrew D. Oxman, Elie A. Akl, Regina Kunz, Gunn Vist, Jan Brozek, Susan Norris, Yngve Falck-Ytter, Paul Glasziou, Hans deBeer et al. 2011. “GRADE Guidelines: 1. Introduction: GRADE Evidence Profiles and Summary of Findings Tables.” Journal of Clinical Epidemiology 64, no. 4: 383–394. https://doi.org/10.1016/j.jclinepi.2010.04.026.

Guyatt, Gordon H., Andrew D. Oxman, Regina Kunz, David Atkins, Jan Brozek, Gunn Vist, Philip Alderson, Paul Glasziou, Yngve Falck-Ytter and Holger J. Schünemann. 2011a. “GRADE Guidelines: 2. Framing the Question and Deciding on Important Outcomes.” Journal of Clinical Epidemiology 64, no. 4: 394–400. https://doi.org/10.1016/j.jclinepi.2010.09.012.

Guyatt, Gordon H., Andrew D. Oxman, Regina Kunz, Jan Brozek, Pablo Alonso-Coello, David Rind, PJ Devereaux, Victor M. Montori, Bo Freyschuss, Gunn Vist et al. 2011b. “GRADE Guidelines: 6. Rating the Quality of Evidence—Imprecision.” Journal of Clinical Epidemiology 64, no. 12: 1283–1293. https://doi.org/10.1016/j.jclinepi.2011.01.012.

Guyatt, Gordon H., Andrew D. Oxman, Regina Kunz, James Woodcock, Jan Brozek, Mark Helfand, Pablo Alonso-Coello, Paul Glasziou, Roman Jaeschke, Elie A. Akl et al. 2011c. “GRADE Guidelines: 7. Rating the Quality of Evidence—Inconsistency.” Journal of Clinical Epidemiology 64, no. 12: 1294–1302. https://doi.org/10.1016/j.jclinepi.2011.03.017.

Guyatt, Gordon H., Andrew D. Oxman, Regina Kunz, James Woodcock, Jan Brozek, Mark Helfand, Pablo Alonso-Coello, Yngve Falck-Ytter, Roman Jaeschke, Gunn Vist et al. 2011d. “GRADE Guidelines: 8. Rating the Quality of Evidence—Indirectness.” Journal of Clinical Epidemiology 64, no. 12: 1303–1310. https://doi.org/10.1016/j.jclinepi.2011.04.014.

Guyatt, Gordon H., Andrew D. Oxman, Victor Montori, Gunn Vist, Regina Kunz, Jan Brozek, Pablo Alonso-Coello, Ben Djulbegovic, David Atkins, Yngve Falck-Ytter et al. 2011. “GRADE Guidelines: 5. Rating the Quality of Evidence—Publication Bias.” Journal of Clinical Epidemiology 64, no. 12: 1277–1282. https://doi.org/10.1016/j.jclinepi.2011.01.011.

Guyatt, Gordon H., Andrew D. Oxman, Nancy Santesso, Mark Helfand, Gunn Vist, Regina Kunz, Jan Brozek, Susan Norris, Joerg Meerpohl, Ben Djulbegovic et al. 2013. “GRADE Guidelines: 12. Preparing Summary of Findings Tables—Binary Outcomes.” Journal of Clinical Epidemiology 66, no. 2: 158–172. https://doi.org/10.1016/j.jclinepi.2012.01.012.

Guyatt, Gordon H., Andrew D. Oxman, Shahnaz Sultan, Jan Brozek, Paul Glasziou, Pablo Alonso-Coello, David Atkins, Regina Kunz, Victor Montori, Roman Jaeschke et al. 2013. “GRADE Guidelines: 11. Making an Overall Rating of Confidence in Effect Estimates for a Single Outcome and for All Outcomes.” Journal of Clinical Epidemiology 66, no. 2: 151–157. https://doi.org/10.1016/j.jclinepi.2012.01.006.

Guyatt, Gordon H., Andrew D. Oxman, Shahnaz Sultan, Paul Glasziou, Elie A. Akl, Pablo Alonso-Coello, David Atkins, Regina Kunz, Jan Brozek, Victor Montori et al. 2011. “GRADE Guidelines: 9. Rating up the Quality of Evidence.” Journal of Clinical Epidemiology 64, no. 12: 1311–1316. https://doi.org/10.1016/j.jclinepi.2011.06.004.

Guyatt, Gordon H., Andrew D. Oxman, Gunn Vist, Regina Kunz, Jan Brozek, Pablo Alonso-Coello, Victor Montori, Elie A. Akl, Ben Djulbegovic, Yngve Falck-Ytter et al. 2011. “GRADE Guidelines: 4. Rating the Quality of Evidence—Study Limitations (Risk of Bias).” Journal of Clinical Epidemiology 64, no. 4: 407–415. https://doi.org/10.1016/j.jclinepi.2010.07.017.

Guyatt, Gordon H., Andrew D. Oxman, Gunn E. Vist, Regina Kunz, Yngve Falck-Ytter, and Holger J. Schünemann. 2008. “Rating Quality of Evidence and Strength of Recommendations: GRADE: What Is ‘Quality of Evidence’ and Why Is It Important to Clinicians?” British Medical Journal 336, no. 7651: 995–998. https://dx.doi.org/10.1136%2Fbmj.39489.470347.AD.

Guyatt, Gordon and Drummond Rennie, eds. 2002. Users’ Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago: American Medical Association Press.

Guyatt, Gordon, Drummond Rennie, Maureen O. Meade, and Deborah J. Cook, eds. 2015. Users’ Guides to the Medical Literature: Essentials of Evidence-Based Clinical Practice. 3rd edition. New York: McGraw-Hill.

Guyatt, Gordon H., Kristian Thorlund, Andrew D. Oxman, Stephen D. Walter, Donald Patrick, Toshi A. Furukawa, Bradley C. Johnston, Paul Karanicolas, Elie A. Akl, Gunn Vist et al. 2013. “GRADE Guidelines: 13. Preparing Summary of Findings Tables and Evidence Profiles—Continuous Outcomes.” Journal of Clinical Epidemiology 66, no. 2: 173–183. https://doi.org/10.1016/j.jclinepi.2012.08.001.

Hindenburg, Carl Friedrich. 1796. Sammlung combinatorisch-analytischer Abhandlungen. Erste Sammlung. Leipzig: Gerhard Fleischer.

Hindenburg, Carl Friedrich. 1800. Sammlung combinatorisch-analytischer Abhandlungen. Zweite Sammlung. Leipzig: Gerhard Fleischer.

Howick, Jeremy. 2011. The Philosophy of Evidence-Based Medicine. Chichester: Wiley Blackwell & BMJ Books.

Howick, Jeremy and Alexander Mebius. 2017. “Randomized Trials and Observational Studies: The Current Philosophical Controversy.” In Handbook of the Philosophy of Medicine, edited by Thomas Schramme and Steven Edwards, 873–886. Dordrecht: Springer Netherlands.

Jadad, Alejandro R., R. Andrew Moore, Dawn Carroll, Crispin Jenkinson, D. John M. Reynolds, David J. Gavaghan, and Henry J. McQuay. 1996. “Assessing the Quality of Reports of Randomized Clinical Trials: Is Blinding Necessary?” Controlled Clinical Trials 17, no. 1: 1–12. https://doi.org/10.1016/0197-2456(95)00134-4.

La Caze, Adam. 2008. “Evidence-Based Medicine Can’t Be….” Social Epistemology 22, no. 4: 353– 370. https://doi.org/10.1080/02691720802559438.

Nordenstrom, Jorgen. 2007. Evidence-Based Medicine in Sherlock Holmes’ Footsteps. Malden, MA: Blackwell Publishing.

OCEBM Levels of Evidence Working Group. 2011. “The Oxford Centre for Evidence-Based Medicine 2011 Levels of Evidence.” Available at https://www.cebm.net/wp-content/uploads/2014/06/CEBM-Levels-of-Evidence-2.1.pdf.

Osimani, Barbara. 2014. “Hunting Side Effects and Explaining Them: Should We Reverse Evidence Hierarchies Upside Down?” Topoi 33: 295–312. https://doi.org/10.1007/s11245-013-9194-7.

Osimani, Barbara and Fiorenzo Mignini. 2015. “Causal Assessment of Pharmaceutical Treatments: Why Standards of Evidence Should Not Be the Same for Benefits and Harms?” Drug Safety 38: 1–11. https://doi.org/10.1007/s40264-014-0249-5.

Stegenga, Jacob. 2014. “Down with the Hierarchies.” Topoi 33: 313–322. https://doi.org/10.1007/s11245-013-9189-4.

Straus, Sharon E., W. Scott Richardson, Paul Glasziou, and R. Brian Haynes. 2005. Evidence-Based Medicine: How to Practice and Teach EBM. 3rd edition. Edinburgh: Elsevier.

Vandenbroucke, Jan P. 2008a. “Observational Research, Randomised Trials, and Two Views of Medical Science.” PLoS Medicine 5, no. 3: e67, 339–343. https://doi.org/10.1371/journal.pmed.0050067.

Vandenbroucke, Jan P. 2008b. “Observational Research, Randomised Trials, and Two Views of Medical Science.” Longer, more detailed version of Vandenbroucke (2008a), available for download at the PLoS Medicine website. https://doi.org/10.1371/journal.pmed.0050067.sd001.

Downloads

Published

2021-04-19

How to Cite

Jerkert, J. (2021). On the Meaning of Medical Evidence Hierarchies. Philosophy of Medicine, 2(1). https://doi.org/10.5195/pom.2021.31

Issue

Vol. 2 No. 1 (2021)

Section

Original Research Articles (clinical research, evidence-based medicine)

License

Authors who publish with this journal agree to the following terms:

  1. The Author retains copyright in the Work, where the term “Work” shall include all digital objects that may result in subsequent electronic publication or distribution.
  2. Upon acceptance of the Work, the author shall grant to the Publisher the right of first publication of the Work.
  3. The Author shall grant to the Publisher and its agents the nonexclusive perpetual right and license to publish, archive, and make accessible the Work in whole or in part in all forms of media now or hereafter known under a Creative Commons Attribution 4.0 International License or its equivalent, which, for the avoidance of doubt, allows others to copy, distribute, and transmit the Work under the following conditions:
    1. Attribution—other users must attribute the Work in the manner specified by the author as indicated on the journal Web site;
    with the understanding that the above condition can be waived with permission from the Author and that where the Work or any of its elements is in the public domain under applicable law, that status is in no way affected by the license.
  4. The Author is able to enter into separate, additional contractual arrangements for the nonexclusive distribution of the journal's published version of the Work (e.g., post it to an institutional repository or publish it in a book), as long as there is provided in the document an acknowledgement of its initial publication in this journal.
  5. Authors are permitted and encouraged to post online a prepublication manuscript (but not the Publisher’s final formatted PDF version of the Work) in institutional repositories or on their Websites prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work. Any such posting made before acceptance and publication of the Work shall be updated upon publication to include a reference to the Publisher-assigned DOI (Digital Object Identifier) and a link to the online abstract for the final published Work in the Journal.
  6. Upon Publisher’s request, the Author agrees to furnish promptly to Publisher, at the Author’s own expense, written evidence of the permissions, licenses, and consents for use of third-party material included within the Work, except as determined by Publisher to be covered by the principles of Fair Use.
  7. The Author represents and warrants that:
    1. the Work is the Author’s original work;
    2. the Author has not transferred, and will not transfer, exclusive rights in the Work to any third party;
    3. the Work is not pending review or under consideration by another publisher;
    4. the Work has not previously been published;
    5. the Work contains no misrepresentation or infringement of the Work or property of other authors or third parties; and
    6. the Work contains no libel, invasion of privacy, or other unlawful matter.
  8. The Author agrees to indemnify and hold Publisher harmless from Author’s breach of the representations and warranties contained in Paragraph 6 above, as well as any claim or proceeding relating to Publisher’s use and publication of any content contained in the Work, including third-party content.
  9. The Author agrees to digitally sign the Publisher’s final formatted PDF version of the Work.