Item — HN Reader

Edit:

Okay, I've read the meta-study now and I think the summary article isn't representing the picture very well. In particular they found for anxiety there actually seems to be evidence in this exact data set that does help.

What they are doing is saying "there isn't 95% evidence it reduces anxiety" therefore "no evidence" even though they mean "some evidence, just not at the statistical significance level" -- it's one of the biggest confusions (and sometimes it feels deliberate) you'll see people do.

Also when you have a confidence interval that big it's a red flag. They themselves admit the data is all over the place.

In summary, don't assume much from the title of the summary article.

2 comments

The data being all over the place on benefits, but pretty clear on harms, is about as good a reason you could want for experts not to recommend something as treatment. That's what it often looks like when something doesn't work, or doesn't work very well. "The error bars are too big to say it works, so we shouldn't tell people it works" is a pretty good thing to inform people about if that's the case.

It's really easy to convince yourself that something works when it doesn't, that's the whole reason why people have to take statistical significance seriously. Maybe it really does work and a really good study could shrink the error bars but that's more hope than anything.

> The data being all over the place on benefits, but pretty clear on harms

Uhhh... no? Did you even read it? This research actually found more benefits than harms. I see it only identified two harms both graded very low.

Let's just quote here the researcher's own conclusions:

"Interpretation There was some evidence that cannabinoids can reduce symptoms of cannabis use disorder, insomnia, tic or Tourette’s syndrome, and autism spectrum disorder, but the quality of this evidence was generally low. Cannabinoids were associated with a greater risk of any adverse events but not of serious adverse events. Overall, there is a crucial need for more high-quality research. Given the scarcity of evidence, the routine use of cannabinoids for the treatment of mental disorders and SUDs is currently rarely justified"

>>> "The error bars are too big to say it works, so we shouldn't tell people it works"

I can see you didn't really understand my comment. There's a huge difference between not saying something is proven to work, and saying it's proven not to work. This study falls in the former category, by the authors own words.

I guess if you weren't around for the 30 years when every marijuana advocate on the planet wouldn't shut up about it being a cure for anxiety, evidence that it is not wouldn't be particularly interesting to you.