▲ 370 points
back
110 comments
Remember to reard the linked editorial for context: https://blogs.bmj.com/bmj/2018/12/13/we-jumped-from-planes-w...
> We conducted the PARACHUTE trial to illustrate the perils of interpreting trials outside of context. When strong beliefs about the standard of care exist in the community, often only low risk patients are enrolled in a trial, which can unsalvageably bias the results, akin to jumping from an aircraft without a parachute. Assuming that the findings of such a trial are generalisable to the broader population may produce disastrous consequences.
>Before you jump to the conclusion that we’re suggesting we jettison RCTs from clinical research, let us clarify that that is not our intention. In an ideal world, new interventions would always be carefully evaluated through rigorous RCTs before widespread adoption. But when pre-existing convictions about an untested intervention affect the population enrolled, even a well conducted RCT can provide misleading results. Without careful attention to context, extrapolating findings from such an RCT to the patient in front of us may be, well, a leap too far.
It's also essential to read the first cited article (https://www.bmj.com/content/327/7429/1459?ijkey=3b8c1a70a8d2...) which very much inspired this paper and is also quite a bit funnier.
I've always known the parachute manufacturers mostly ride on fear, uncertainty and doubt on this. They ruthlessly cash in on people's lack of knowledge and fear of death, advertising parachutes as a safer option which people buy because it's like insurance; if you'd happen to fall, you'd at least want to feel having prepared.
So I'm glad to hear there's now some scientific evidence pointing to the futility of parachutes. Surely, the experiments were carried out in a constrained environment with some unavoidable assumptions baked in but it does rather convincingly suggest that parachutes consistently do not offer any help in the event of freefall.
A corollary might appear that parachutes might even do more harm, adding weight to the falling person which could cause further injuries unnecessarily upon touchdown.
I'll look forward to follow-up research.
The big question is how would altitude affect the results.
For experiments at higher altitudes they might need to locate an airstrip situated on a mountain, with enough runway to allow larger planes to land there and enough free space on the tarmac for several planes while they carry out the jumps. It will likely shed much more light and build confidence to the applicability of this new research and the obvious conclusion.
Also, I'd like to see jumps done from different kinds of aeronautic vehicles, by more people, different brands of parachutes etc if should there be any differences regarding that. But it looks like after a few rounds of serious experiments we should quickly conclude the benefits are thin and see no need for excess research but, rather, reaching a steady conclusion.
I suspect that landing on water has no discernible effect on the health of the suspect. And he will be completely soaked, so avoid jumping over water and always try to land on land.
A coworker of mine once made this proposal. 5 of us would start a company. The company would buy key man insurance for everyone, then go on a company team building trip. On this trip, we'd all skydive, but one of the 5 parachutes would be non-functional...Profit!
This coworker of mine soon after left the company by storming out of the meeting room and slamming the door, then took up playing online poker for a living.
Key man insurance policies explicitly exclude parachute jumps.
I know because one of the founders of the company I work for used to jump, and he had to give up his hobby for that reason.
I think you'll find that key-man insurance policies contain clauses explicitly to prohibit these sorts of shenanigans.
I guess that was around 2005? What does he do now?
This isn't just for giggles. It's evidence that when there's a high perceived risk, "normal" people don't sign up for trials and it skews the results.
> The study also has several limitations. First and most importantly, our findings might not be generalizable to the use of parachutes in aircraft traveling at a higher altitude or velocity. Consideration could be made to conduct additional randomized clinical trials in these higher risk settings. However, previous theoretical work supporting the use of parachutes could reduce the feasibility of enrolling participants in such studies.16
> 16. Newton SI. Law of Universal Gravitation.Philosophiæ Naturalis Principia Mathematica, 1687.
>Compared with individuals screened but not enrolled, participants included in the study were on aircraft at significantly lower altitude (mean of 0.6 m for participants v mean of 9146 m for non-participants; P<0.001) and lower velocity (mean of 0 km/h v mean of 800 km/h; P<0.001).
This is an amusing didactic example of how the "fine print" can invalidate a study's conclusion.
In case anyone else got curious about parachuting injury statistics, Google found me this (also cited by the article):
"Parachuting injuries: a study of 110,000 sports jumps", 1987, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1478603/
All 6 deaths they observed, were either failures to activate the parachute (4 cases), or parachute malfunction (2 cases).
> PArticipation in RAndomized trials Compromised by widely Held beliefs aboUt lack of Treatment Equipoise (PARACHUTE)
That has got to be one of the most torturous acronyms I've ever seen. That's a bit of a stretch.
Oh my sweet summer child...
Biomedicine has gone whole hog into torturing acronyms to make useful sounding cohort names.
Would you indicate this is a satirical article? Half the time I was looking for jump height or understanding if adding parachutes to commercial aircraft that suddenly experienced breakage such as a missile would improve passenger survival.
The fact that this isn't immediately tagged as satirical and requires actually reading the article and fully understanding the context (rather than just reading the headline and making a quick assumption) is pretty much the entire point of the "study" and the authors of it. Marking it as satirical would sabotage the purpose.
Anecdotally I find the slow realization that something is satire to be an integral part of it's entertainment value.
I've done over 900 jumps so I can fill you in, no references ATM because I'm on my phone.
If you jump with a parachute, you will see chance of death go from 0% at zero feet, to approximately 100% at 35 feet. From there, it will remain near 100% until you reach about 600 feet, which is about the minimum distance it takes a parachute to open. Above 1000 feet and chance of death floors out near 0%
There is obviously an element of satire, but are the authors are also trying to make some sort of statement about clinical trials?
Jump height is even in the abstract. You have to read it though, skimming won't work.
"The PARACHUTE trial satirically highlights some of the limitations of randomized controlled trials."
Anyone know if these results have been reproduced by anyone? I'm a bit skeptical.
Assuming you’re serious, you can reproduce this yourself. Put on a backpack, stand on a chair and step off. The trial participants “jumped” from 0.6 meters.
I wonder if they where gunning for an Ig Nobel Prize. I'm sure they'll be nominated.
> A key strength of the PARACHUTE trial was that it was designed and initially powered to detect differences in the combination of death and major traumatic injury. Although the use of softer endpoints, such as levels of fear before and after jumping, or its surrogates, such as loss of urinary continence, could have yielded more power to detect an effect of parachutes, we believe that that our selection of bias-resistant endpoints that are meaningful to all patients increases the clinical relevance of the trial.
I love how much detail into which they go.
But not a placebo-controlled one? Shame.
Their methodology has some serious shortfalls ;)
I think this is like the medical research paper version of the famous RFC for IP over Avian Carriers.
> The PARACHUTE trial satirically highlights some of the limitations of randomized controlled trials. Nevertheless, we believe that such trials remain the gold standard for the evaluation of most new treatments. The PARACHUTE trial does suggest, however, that their accurate interpretation requires more than a cursory reading of the abstract. Rather, interpretation requires a complete and critical appraisal of the study.
I love this paper. Whenever you see pop news articles saying "Study shows X is good/bad for you," if you dig in to the paper, it's often something like this. When you read the specific details, you realize the study doesn't generalize to "You should/shouldn't do X." But not enough people read the details, so it gets circulated into conventional wisdom :(
Must have been a fun IRB approval process. Wish we could also see those applications/disclosures!
>Ethical approval: This research has the ethical approval of the Institutional Review Board of the Beth Israel Deaconess Medical Center (protocol no 2018P000441).
Immediately sprung to my mind: http://www.lukesurl.com/archives/comic/444-fall-in-all
I know this is a boring response, but I feel like there’s a formalism here to consider why a RCT would be “obvious.”
Let’s say you’re going to use some causal model, like a regression adjustment technique. You could for example assign people to the treatment group (receives parachutes) and the control (no parachutes), and then observe who lives and dies, as well as a bunch of potential confounders like altitude, age, fitness, whatever.
Fit a logistic regression to predict the outcome (survival) based on the treatment (parachute) controlling for the other characteristics. Then read off some effect size and ststistical significance.
Or better yet, and here’s the important part, you could make it a Bayesian logistic regression by considering prior distributions for the regression model’s fitted coefficients, and sampling draws from the posterior distribution of coefficients using the data set and your priors.
So what is the prior on the coefficient for the treatment term (parachutes)? Well, probably pretty damn high. Definitely some strongly informative prior, take your pick of historical data or effectiveness rates of physical safety equipment, whatever.
From this prior, and making some neutral assumptions via the priors on other weights, you could figure of what the effective sample size would be for a data set to disconfirm your prior (e.g. a posterior with a mode on the parachute coefficient far away from your strong prior). Sort of like a power analysis, but assuming a fake data set that shows nothing but failed parachutes. How much of that silly data would you need based on your prior?
What this would tell you is that you’d need some insane, physically ludicrous amount of data that flies in the face of an obvious prior, that what would be the point of running the study? You’re just going to confirm your prior.
So the real question is how often is this a realistic description of other situations when you want to study a treatment?
That’s the thing, right? That the author kind of wants to be snarky about.
But really, it’s pretty fair to say you don’t have such a strong prior that the study would be futile, even in cases when you sort of do feel like the conclusion is obvious (e.g. taking Tylenol leads to less pain, college kids prefer drinking instead of homework). While it passes some gut test of what’s obvious, that’s different from really betting on such a one-sided prior that a study is futile.
To me it suggests most of the sort of “duh” RCTs carried out are pretty much fine. Whether or not the study is worth it or is informative would be based on other priorities like cost, licensing or certification requirements, whether it’s of value to specialists who care about splitting hairs on accurate effect size measurement, etc.
N=92 participants is not big enough of an N to conclude anything. To see differences you're going to have to go up to 10 times more at least.
I'd hate to be part of the control group.
When can we expect a news article telling people scientists advise not to wear parachutes?
Some comments have touched on a few of these points but I’ll try to bring them together. So first, yes the headline is a joke (BMJ Xmas has satirical content), but it also has a history beyond the immediate point of the paper—that sometimes RCTs can’t accrue properly to a control arm to fairly evaluate an intervention.
Quick points:
- There’s a history to this analogy and paper within
biomedicine and this journal (BMJ) from a 2003 article,
which I’ll get to.
- Parachutes are a dangerous metaphor in medicine,
where almost nothing has an absolute risk reduction
of >99% (note: not 100% because, yes, a handful
of people have survived falling from altitude without
a parachute), especially over the time-frame
of a matter of hours.
- This should not be a call to stop attempting RCTs
(which is the conclusion some commenters have made),
but an exhortation to find ways to create better ones
when conditions are challenging. Frequently, objections
to doing an RCT because “how could we withhold X
from the control arm!”, are not as obvious once
the data are in.
The broader point is that the 2003 “parachute” article in the same journal (BMJ), was frequently incorrectly understood / used.
While “parachutes” makes an easily understandable headline, it is almost totally unrelated to the field of medicine where we rarely have a shot at doing something as obviously lifesaving as making someone hit the ground at 10 mph vs 120 mph. The problem is that people have cited the 2003 paper mentioned in the thread to justify a number of interventions that ended up not being better than prior care. The interventions were started in good faith because was “obvious” to their creators that doing X would be helpful (spoiler: it usually wasn’t or wasn’t that beneficial).
A lot of this is cribbed from Vinay Prasad, who has a twitter thread about this: https://twitter.com/VPplenarysesh/status/1073298754298556416
He is a controversial figure, but I think he does a good job of hammering home some important skepticism about a great deal of medical literature and practice to a broad audience.
I’d appreciate hearing objections to the above, btw.
Please fix the title. It's misleading. The linked article in no way claims that "Parachutes prevent death when jumping from aircraft".
The current title "Parachutes prevent death when jumping from aircraft: randomized controlled trial" is exactly the opposite of the article's conclusion: "Parachute use did not significantly reduce death or major injury (0% for parachute v 0% for control; P>0.9)." Nor is it the title of the actual article.
This title has been editorialized from
"Parachute use to prevent death and major trauma when jumping from aircraft: randomized controlled trial"
to
"Parachutes prevent death when jumping from aircraft: randomized controlled trial"