▲ 127 points
back
77 comments
For those not in Academia, it's worth noting that when you get to his level, it's more like being a VP or senior leader. Ultimately, yes you are responsible for the quality of output of your team, but you are absolutely not looking in detail at every paper, more the ideation and focus of the department.
He should be grilling his assistant profs / research fellows to get their act together and raise the bar, but this doesn't show malfeasance.
Important point, thank you.
It doesn't mean we shouldn't work to fix the systemic issue though. It may not he his "fault", but how do we improve our system as a soceity so we put ourself on a upward trajectory not the seemingly downward trajectory we've been on lately.
I'm glad the GP pointed this out, because nobody talks about it. The "PI" of a lab is manager, so these issues are like an accountant embezzling money without the manager knowing. Maybe the manage should have put better processes and monitoring in place, but they were trusting their team members to behave properly, which is not unreasonable IMO.
The real issue is that the people who conduct the fraud are usually grad students or postdocs, whose entire future depends on the success of their research project. Fake results are pretty much guaranteed.
Think about it this way: Imagine you have a lab group with 10 PhD students, each with their own hypothesis to investigate (e.g. intervention A will reduce the rates of disease B in mice). What are the odds that all 10 students will prove their hypothesis and generate publishable results? No way it's 10/10 obviously...it's more like 5/10. So what is supposed to happen to those 5 students that were tasked with investigating the bad hypotheses? Our current system implicitly penalizes these students to the extent that their careers in academia are over, and even earning their PhD is not certain. BTW, I know I am being overly general here, and the student will likely have several parallel projects on-going, can pivot to other things, etc, but hopefully my general point is clear.
I had a bit of a eureka on this subject this afternoon: when looking at scientific fraud and who to blame, we (as a society) tend to focus on who stands to gain if the fraud is successful, but instead we should look at who stands to lose the most if the fraud is caught.
Let me explain via a 2x2 matrix (which I highly doubt will render properly, but here goes):
Actor | Fraud is successful | Fraud is caught
------------| -------------- | ----------------
Professor | Scenario A | Scenario B
Student | Scenario C | Scenario D
Scenario A: If a fraud is successful, the senior author gets a small benefit in the form of a slight raise, incremental increase in success rate of next grant, maybe an award, some endorphins from the praise. Very minor actually.
Scenario B: If the fraud is caught the senior author's career could be in shambles, like resigning from their tenured position, losing investors in spin-offs, humiliation, etc. They have a lot to lose.
Scenario C: In the event of a successful fraud, the student stands to gain a lot in the form of job prospects, future income, and generally accomplishing their life's ambitions. There is a huge payoff for the student in this scenario.
Scenario D: If they don't perpetrate fraud (to salvage a bad result), their career in academia is over, they have wasted 3-4 years of the life, which is the same outcome as if they did perpetrate fraud and got caught. The student has nothing to lose!
Worth remembering that Linus Pauling went off the deep end on Vitamin C. Not that he faked stuff, but that even very smart people get sucked into wierd science sometimes.
So leaping from this, to Semanza is emperor's-new-clothes is pushing too far. If he's moved into a space where marginal behaviour around research is normalised thats really bad but it doesn't have to mean his Nobel wasn't warranted, that depends on other questions: Questions which might still need to be asked, but this isn't the smoking gun. It's maybe more why asking those questions might become more important.
Einstein's discovery of relativity is something everybody takes for granted now, but think about how exceptionally ridiculous the notion that light travels at the same rate of speed from all perspectives is; or how the universe will bend over backwards, including having people 'advance through time' at different rates of speed or literally physically change the distance between things, in order to maintain this perception. It's like some sort of a simulation filled with spaghetti code, yet it's our real universe and has been as 'confirmed' as anything can be in science.
But... imagine it wasn't. If you had a 'logical and normal' understanding of the universe, and heard of somebody researching something with these sort of implications, then a normal person would probably think they had simply lost their mind. Calling it just "weird" would be a dramatic understatement.
We always have, and probably always will have, an arrogance of the present. It's explained by a tautology - what we think is right, is what we think is right. Or otherwise, we wouldn't think in such a way. Yet the entirety of human history shows that sooner or later, much if not most of what we think we know to be true, will be proven to be simply wrong. Sometimes it will be in the 'needs refinement' bucket, but much of it will also just end up in the completely and fundamentally wrong bucket. And the scientists willing to indulge 'weird science' are those who will take us there, because you're not just cleanly stepping from one branch to another A->B->C, but just completely scrapping the tree and starting with an entirely new root node.
Einstein did not discover that the speed of light was fixed. This was a consequence of Maxwell's research on electromagnetism in the late 19th C.
It is entirely reasonable for any physicist (indeed, Maxwell himself could've done it with enough time), to work through the consequences of a non-relativistic equation for the speed of light.
This approach is routine in physics, and there's nothing in it that would appear crazy or unusual. Physicsts do not simply just pluck crazy ideas from their imagination and, from-nothing, deduce possible consequences. This is a myth, and insofar as people do that, they're cranks.
The imagination of a phyiscist is heavily guided by the totality of physics that has been developed. Einstein did not simply say, "what if the speed of light was fixed?"
He said, 'given we have good reasons to suppose it is, what are the concequences on relativity ?'
> We always have, and probably always will have, an arrogance of the present. It's explained by a tautology - what we think is right, is what we think is right.
I think it's culture + semantics/semiotics. I see it occurring most frequently in the form "What is true, is what we believe (~'consensus') is true" (the word belief is usually not explicit though).
In practice, the word "is" has multiple meanings, one of them being "[It is my opinion that] X is true", and pointing out the presence of the implicit opinion part is culturally unacceptable, depending on the scenario. It's a lack of cultural distinction between the universe and reality, they are typically conflated in this era, in our geographical region. Personally, I think LLM's offer some compelling insight into the process underlying this phenomenon, and I bet this can be proven out.
I agree it's the scientists who take us to weird places that advance the needle (and something that's discouraged in academia, but that's a different conversation).
However there's a big difference between proposing something exceptionally ridiculous and fraud leading to retraction
simulation filled with spaghetti code, typed out by our god and his noodley appendeges.
It's a pretty common thing, too [1].
Concerning Nobel disease:
Couldn't is be that what makes (some) Nobel-prize-worthy discoveries possible is an openness with respect to "crazy" ideas, of which some small amount actually turn out to be right, and thus lead to Nobel prizes?
So, I think these Nobel prize winners just continue as before: thinking about and exploring "crazy" ideas that might have a huge impact if they turned out to be right. It's just that the next times, it will be the very common situation that these crazy ideas won't be right?
Feels like a case of Berkson's bias[1] to me.
I went deep down the Pauling Vitamin C rabbit hole once and got to point out that many studies not seeing effects are not actually giving people gram-megadoses, but mg-homeopathic ones. This story might not be as closed as some podcasters and other influencers pretend. Maybe it’s not double nobel laureate Pauling who was so wrong that he has pretty much become a quack in popular knowledge, but the fields of nutrition and perhaps medical science that are shoddy.
In fact nutrition and medical science are quite well known to be some of the worst offenders when it comes to bad methods and scientific misconduct, particularly in the past few decades (as also shown by OPs link).
Also - like apparently many of those perpetuating the story - I got my initial opinion about the Vitamin C topic being quackery from Wikipedia, but know better now not to generally trust it for medical topics since it’s quite well known that marketing departments of the pharmaceutical industry have a lot of time on their hands to write articles that benefit them. I personally burned myself with a safe&effective&local medical product promoted there with scientifically sounding terms with all criticism erased or “debunked” around 2014 and have permanent eye damage now (there has been a class action lawsuit few years later).
But to add to the nobel-laureate-turned-venturous trope, here is a recent example from physics: https://en.wikipedia.org/wiki/Brian_Josephson
I would say these people generally like to follow wherever curiosity leads them, without giving much thought about peer opinion, which is why they are the ones winning prizes for revolutionizing discoveries. They are freer to do so once the price is in and this may lead them any direction.
A milligram dose is not usually considered homeopathic. A 6C homeopathic dilution, which is on the less potent end of homeopathic medicines, is 1:10^12.
Now, a gram vs. milligram would be between 2C and 3C (or 3X on the decimal scale) so it can be described on a homeopathic scale, but then again, full-strength of 0X or 0C can also be on that scale so I don't think this is the interpretation you mean.
FWIW, "Mega-dose vitamin C in treatment of the common cold: a randomised controlled trial" at https://onlinelibrary.wiley.com/doi/abs/10.5694/j.1326-5377.... uses 1g and 3g doses and refers to the 30mg dose as "placebo". It's apparently hard to get the taste right with truly homeopathic doses.
> many studies not seeing effects are not actually giving people gram-megadoses
Pauling is seen as a quack when it comes to vitamin C because he claimed it could help treat all sort of things; for colds, for cancer, for AIDS treatment, for asthma, for mononucleosis, and for far, far more, as he uncritically lists every single positive connection to vitamin C in his 1987 book "How to live longer and feel better". https://archive.org/search?query=%22How+to+Live+Longer+and+F...
That makes it hard to know what studies you refer to, and certainly there are studies which did use gram-megadoses and failed to replicate or find support Pauling's findings, like:
Anderson TW et al, in "Vitamin C and the common cold: a double-blind trial", Canadian Medical Association Journal (1972) used 1-gram megadoses, with 4-gram megadoses at the onset of colds. "It was found that in terms of the average number of colds and days of sickness per subject the vitamin group experienced less illness than the placebo group, but the differences were smaller than have been claimed and were statistically not significant". https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1940935/pdf/can...
Pauling's original study, with a smaller population and shorter time frame, suggested the benefits were far more significant, so should have been visible in that Canadian study, and that's only one of several >= 1 gram studies.
> Maybe it’s not double nobel laureate Pauling who was so wrong that he has pretty much become a quack in popular knowledge, but the fields of nutrition and perhaps medical science that are shoddy.
Do you think double-Nobel-Prize-winner Pauling was right that megadoses (10g) of vitamin C could treat cancer?
Because that was tested. "Failure of High-Dose Vitamin C (Ascorbic Acid) Therapy to Benefit Patients with Advanced Cancer — A Controlled Trial", https://www.nejm.org/doi/full/10.1056/nejm197909273011303 with 10 g per day.
Oh, but wait - Pauling then said vitamin C only worked on cancer patients who had not received already had chemotherapy.
Nope, still not the case: "High-Dose Vitamin C versus Placebo in the Treatment of Patients with Advanced Cancer Who Have Had No Prior Chemotherapy — A Randomized Double-Blind Comparison", https://www.nejm.org/doi/full/10.1056/nejm198501173120301 with 10 g daily.
Oh, but the problem is the Mayo Clinic doesn't follow Pauling's protocol, and didn't continue treatment beyond 75 days. https://www.science.org/doi/10.1126/science.1925592 .
When should other people stop listening to Pauling's claims?
So Pauling and Cameron did their own study to support the claim ... which was criticized for the lack of blinding and poor selection of controls. (ibid).
> it’s quite well known that marketing departments of the pharmaceutical industry have a lot of time on their hands to write those articles that benefit them
It's quite well known that people in the pharmaceutical industry, and their friends and family, also get colds, cancer, and more. Do you really think they are hiding a cure from their co-workers, friends, and family?
It's also well-known that the supplement industry makes billions of dollars, among other things, from actual homeopathy, and this money gives them lots of time to write those articles that benefit them.
Agreed about Josephson!
John Clauser has gone off the deep end with his climate denialism.
And William Shockley with eugenics
One has to wonder if the Nobel prize was a mistake. Even if it was, no , a Nobel prize can't be revoked as per this[0]
[0]https://www.nobelprize.org/frequently-asked-questions/#:~:te....
Lucky for this guy
https://www.nobelprize.org/prizes/medicine/1949/moniz/facts/
In many ways though, I think it's a strength that such a prize cannot be revoked, because it emphasizes the zeitgeist of a time. I think it can help provide not only some humility to us in a different era, but also perhaps help us learn more from the past.
As an interesting factoid, lobotomy was practiced at all levels of society at the time. Even JFK's sister was secretly lobotomized [1]. It was covered up for years as it went about as well as one would expect a lobotomy to go.
But Moniz's case isn’t a case of fraud , no? I’m not insinuating this case is fraud fyi
How do you accidentally have 3 images that are exactly the same?
Only half joking, but have you ever had the issue on a Mac where you copy something and it randomly fails to copy, so when you paste, it pastes the previous copy?
Easy to imagine, slack that it did not get seen.
(I think the bug is at least 15 years old).
Or if you have 20 files in a folder called IMG0887.jpg, IMG0897.jpg, etc, and you need to open them one after the other with an open file dialog, easy to pick the wrong one. A scientist isn't necessarily IT smart, particularly in medicine. Not saying it's what happened though.
Seems to happen a lot when I'm pasting in/out of a browser; not sure if that's the same issue, or one further augmented by Firefox? :)
wait it does not happen just to me? is it widespread? I thought I was going mad or my keyboard starting to break down!
With multiple people working on a paper, some junior, some senior, tight deadlines, and last minute changes, honest mistakes are certainly possible.
Paper figures are usually hand assembled using Adobe Illustrator or similar, which can be error prone.
For instance, placeholder images might have been added during drafting, and they were not later replaced.
So my assumption is the figures must be vital data filled figures. If they're just diagramatic ones (fig 1 type) that are similar amongst articles from the same group on thr same line of work then it's probably plagiarism by the letter of the law but not the spirit of the law.
Remember the Sokal Hoax and Sokal Squared?
Turns out that STEM also has all those problems.
For profit universities pushing academics to "publish or perish" are the real enemies here, for all disciplines.
Publish or perish and other academic pressures are hardly unique to for profit or even non profit universities, even Government institutions have similar incentive models to get tenure or other advancements, it seems pretty universal ?
Do you know of a study or citation about any academic research quality/quantity challenges that are found at a higher rate in for profit / non profit private institutions ?
Tenure = money
I'm pretty sure that this Nobel Prize winner does not work at a for profit university.
He works at Johns Hopkins, which is a private university that works closely with a health system called Johns Hopkins and the Johns Hopkins hospital (the medical school). When you say "for profit", be aware that they definitely take money from the government and patients and use it to develop new intellectual property that generates large amounts of money for them (although I guess the "profit" from that doesn't benefit the shareholders).
These "non-profits" have multi-billion dollar budgets and are highly "profitable" depending on how you like to define it. https://www.nytimes.com/2020/02/20/opinion/nonprofit-hospita... In these cases, the non-profits are given tax incentives provide a certain amount of community benefit.
For what it's worth, I sort of read that as "universities incentivized to maximize profit or income", not literally formally for-profit universities. Sort of a biting remark about typical university motivations, not literal.
Even if the poster didn't intend it that way, I think it could easily be read that way.
Journals are also absolutely involved and incentivised.
Just because it's non-profit doesn't mean it's not about money.
This is true but a critical difference is that STEM is a lot more falsifiable.
Postmodern deconstruction lends itself to a million interpretations, but biological testing either works or doesn't, and inconclusive generally means it doesn't.
> Remember the Sokal Hoax and Sokal Squared?
The Sokal Hoax, while famous, is not very notable. They did not even publish in a peer reviewed journal.
Publish or perish is primarily about peer competition and promotion, not pressure from admins.
Teaching is the money maker.
In order to get tenure people need to have published.
Keeping track of retractions is good, but I'm on the fence as to whether the community should be making it a shameful act (like committing a crime), or a positive one (like apologising, and admitting mistakes).
We want scientists to have incentives to not deliberately commit fraud, but also feel that they can rectify mistakes.
Having retraction be shameful might do the former, but will harm the latter.
I don't have an answer off the top of my head, this is just my observation on this.
Honorable retractions, as a result of honest errors, happen all the time in science and, while unfortunate, are not a huge deal.
Much different are the retractions based on research fraud. retractionwatch provides a valuable service in tracking all types of retractions, and in warning the community about unethical or predatory journals.
I like your term "honourable retraction".
But if people and retraction watch don't differentiate between these and dishonourable ones, then it doesn't really help.
there are a lot of "bad science" problems and people tend to get them mixed up.
Here's a list of all the things I can think of, roughly ordered by my opinion of seriousness:
1. Fraudulent data in real publications in real journals, even sometimes the most prestigious journals like Science, Nature, Cell. (cases in article are this situation.) Worst of all is when it has a real impact on policy, or leads to bad investment in a new technology that can't work.
2. P-hacking, questionable research practices, etc. without fraud, by "legitimate" academics. This happens everywhere but seems to be a bigger problem in social sciences especially branches of psychology. Real-world impact has been less severe but does cause problems and wasted money sometimes. The worst impact is probably that it causes some young researchers to waste their careers on dead-end lines of research.
3. Really badly done studies to legitimate dubious supplements. Subcase of 2. I actually rate these as less severe, because the average consumer of dubious supplements is not reading these pointless studies, and supplements are basically unregulated, so I don't think getting rid of these studies would make much difference.
4. Completely fake garbage papers in paper mill journals. Not just fake data but outright total nonsense. These are ignored by most scientists, so less severe, but are a concern for the people who fund scientists, and they can cause real trouble if they end up in a meta-analysis.
I only write this list because so much of the discussion of fraud and bad research seems to conflate all of them together. But they're all distinct.
I had mentally accepted p-hacking and paper mill garbage, but seeing Nobel-prize-winning labs publish photoshopped blots in top journals has shaken my faith in our scientific institutions far more.
5. HARKing: "Hypothesis after results known" is also a problem to produce even "legitimate" papers by discarding the original hypothesis after negative findings and come up with a new one that fits the data. Its similar to p-hacking and also skews science into "promising" dead-ends.
> but seeing Nobel-prize-winning labs publish photoshopped blots
And they were caught because they edited poorly which could be detected automatically or by human eyes. Think about well versed image manipulators or generative AI...
I think this erosion of trust is the worst, AI can produce and it will propably come.
"A Nobel prize-winning researcher whose publications have come under scrutiny has retracted his 10th paper for issues with the data and images.
Gregg Semenza, a professor of genetic medicine and director of the vascular program at Johns Hopkins’ Institute for Cell Engineering in Baltimore, shared the 2019 Nobel prize in physiology or medicine for “discoveries of how cells sense and adapt to oxygen availability.”