Item — HN Reader

(2020) Will COVID-19 be evidence-based medicine’s nemesis?

https://journals.plos.org/plosmedicine/article?id=10.1371/jo...

"It is surely time to turn to a more fit-for-purpose scientific paradigm. Complex adaptive systems theory proposes that precise quantification of particular cause-effect relationships is both impossible (because such relationships are not constant and cannot be meaningfully isolated) and unnecessary (because what matters is what emerges in a particular real-world situation). This paradigm proposes that where multiple factors are interacting in dynamic and unpredictable ways, naturalistic methods and rapid-cycle evaluation are the preferred study design. The 20th-century logic of evidence-based medicine, in which scientists pursued the goals of certainty, predictability and linear causality, remains useful in some circumstances (for example, the drug and vaccine trials referred to above). But at a population and system level, we need to embrace 21st-century epistemology and methods to study how best to cope with uncertainty, unpredictability and non-linear causality [16].

In a complex system, the question driving scientific inquiry is not “what is the effect size and is it statistically significant once other variables have been controlled for?” but “does this intervention contribute, along with other factors, to a desirable outcome?”. Multiple interventions might each contribute to an overall beneficial effect through heterogeneous effects on disparate causal pathways, even though none would have a statistically significant impact on any predefined variable [11]. To illuminate such influences, we need to apply research designs that foreground dynamic interactions and emergence. These include in-depth, mixed-method case studies (primary research) and narrative reviews (secondary research) that tease out interconnections and highlight generative causality across the system [16, 17]."

table1: https://journals.plos.org/plosmedicine/article?id=10.1371/jo...

1 comments

Sounds extremely susceptible to quackery and hand-waving appeals to authority.

The author of the article was in favor of face masks even in the early stages of the COVID pandemic. Meanwhile, the mask opponents concluded from the principles of evidence-based medicine that masks are not effective. During a pandemic, there is no time to wait for 10 years until sufficient evidence accumulates. If evidence-based medicine cannot be used in the case of pandemics, an improved methodology is needed.

Whether or not "guesswork that's probably scientifically informed" is in fact "improved methodology" is the question. As is using the fact that heads came up when you flipped a coin hoping for heads is sufficient grounds for applying the same method to much more consequential tasks. I for one am very glad we didn't abandon the supposedly far too cautious approach of evidence-based medicine when we properly tested the COVID vaccines, for example, before administering them to billions. That of course is far more consequential to ending the pandemic than masks ever were. But sure, for low-effort low-risk policies like "wear masks made out of whatever, it might do something", we can go with scientifically informed guesswork.

You can check the author ( Trisha Greenhalgh ) other articles:

"But there is a more fundamental—i.e. philosophical rather than methodological or practical—objection to the emphasis on RCTs to the exclusion of other kinds of evidence, and that is the assumption, based on what might be called naive empiricism, that data can be identified, collected, analysed and summarized without the need for theory. Academics in many other scientific disciplines emphatically reject the assumption that controlled experiments should always and necessarily over-ride mechanistic evidence, defined as evidence produced by multiple different methods which help illuminate and explain phenomena at a theoretical level [40,41]."

https://royalsocietypublishing.org/doi/10.1098/rsfs.2021.001...

or ..

"Adapt or die: how the pandemic made the shift from EBM to EBM+ more urgent "

https://ebm.bmj.com/content/27/5/253.full

"Evidence-based medicine (EBM’s) traditional methods, especially randomised controlled trials (RCTs) and meta-analyses, along with risk-of-bias tools and checklists, have contributed significantly to the science of COVID-19. But these methods and tools were designed primarily to answer simple, focused questions in a stable context where yesterday’s research can be mapped more or less unproblematically onto today’s clinical and policy questions. They have significant limitations when extended to complex questions about a novel pathogen causing chaos across multiple sectors in a fast-changing global context. Non-pharmaceutical interventions which combine material artefacts, human behaviour, organisational directives, occupational health and safety, and the built environment are a case in point: EBM’s experimental, intervention-focused, checklist-driven, effect-size-oriented and deductive approach has sometimes confused rather than informed debate. While RCTs are important, exclusion of other study designs and evidence sources has been particularly problematic in a context where rapid decision making is needed in order to save lives and protect health. It is time to bring in a wider range of evidence and a more pluralist approach to defining what counts as ‘high-quality’ evidence. We introduce some conceptual tools and quality frameworks from various fields involving what is known as mechanistic research, including complexity science, engineering and the social sciences. We propose that the tools and frameworks of mechanistic evidence, sometimes known as ‘EBM+’ when combined with traditional EBM, might be used to develop and evaluate the interdisciplinary evidence base needed to take us out of this protracted pandemic. Further articles in this series will apply pluralistic methods to specific research questions."