Item — HN Reader

"The prolific chemist, who has published a study every 37 hours this year..."

I found this part quite impressive until I read the following part:

"Luque is constantly publishing papers. Last year he authored some 110 articles. So far this year he has published 58. The chemist admitted that since December, he has been using the artificial intelligence program ChatGPT to “polish” his texts. “These months have been quite productive, because there are articles that used to require two or three days and now I do them in one day,” he said."

Even if this is unrelated to the actual story, I find it a tiny bit disturbing. Is this the new standard practice? Let some robot spew out paragraphs of text to convey the scientist research?

14 comments

It speaks to the broken state of measuring scientific and academic output. It's 99% a charade, people churning out machine generated papers nobody reads but get quoted in an attempt to justify the inflated claims of other machine generated garbage papers, in a circular attempt to fudge the stats that decide the funding and promotion.

The idea that someone can even read, edit and make co-author level contributions in papers aimed for a top-level journals, all in the span of 37 hours, is utterly ridiculous.

It's basically the equivalent of SEO spamming the Page rank algorithm with crummy pages with many back-links and repeated search terms. The search optimized term here is the author's name.

Why maximize paperclips when terms like "vegetative electron microscopy" could be trending in the literature.

For those who didn't read the article, or for those who did, but didn't follow its link to the PubPeer thread containing the "vegetative electron microscopy" observation:

https://pubpeer.com/publications/7C9F0CCD493B1129135A3A918B0...

Search the page for the term and you'll see the relevant comment.

The term seems to stem from an OCR cluster-mishap, where a valid scientific document was once OCR'ed incorrectly. The page has 2 columns of text and the left column has a running sentence 'ending' with "vegetative" before continuing on the next line in the same column. But at the same height in the column on the right a sentence continues, visually starting with "electron microscopy".

The term is entirely meaningless in the field.

This isn't just "using a little ChatGPT to brush up my English" at all.

Other comments point out that the contents of abstract and article don't even match, the abstract being about bacteria but the article contents about gene delivery...

I don't need convincing that ML will have tremendous positive impact in math and science disciplines, it's obvious, but the way ML was used in this case is not a defensible usage.

Yeah, from the article I get the sense that he is often agreeing to add his name to papers for network affect, perhaps even as the main author.. So his own criteria for what he writes may be factual, but seems a bit off topic for what a reader should expect on seeing his name.

To be free of this pain.

TBH I'm terrified of the double-whammy effect of researchers becoming less and less competent as AI takes the reins (a la Wall-E) and good science getting drowned out by crap.

Modern academia incentivizes quantity over quality, and reviewers for all but the most prestigious journals don't have sufficient time or expertise to properly review submitted papers. So this does not surprise me.

From my experience most researchers don't actually read most papers, we just don't have time. The figures are what tells the real story, not necessarily the interpretation of the authors.

I don't disagree with you, but this is a horribly sad comment. I worry that the text of papers often gives short shrift to nuance and subtlety that is necessary for reasonable interpretation.

I once read a lot of medical papers -- probably a few hundreds. There was often no real connection between the reported data and the conclusion. Sometimes the conclusion was along the lines of "a weak relation between X and Y was found" but the data showed a strong relation. Sometimes it was the opposite. Sometimes the conclusion was that X was good but the data showed X was bad. Sometimes it was the opposite.

It was almost as if many of the authors couldn't do the kind of math expected of high schoolers that apply to university to study science.

My takeaway was 1) to deeply distrust doctors as scientists (and as people who could think) and 2) mostly ignore the text surrounding the tables and graphs and just go straight to the data.

Physicians are definitely not scientists, at least the majority of MDs. MD is not a research degree, it's a memorization degree

A couple of years ago I spoke to my near-phd biochem friend and discussed this problem with him at length. I have since come to believe that what science needs is a git of science. Aka, a scientific way to build science over from scratch and explore different approaches and research in a version control kind of way. A property of science is that existing research should be immutable, so you can see the natural progression over time, so for this I thought a blockchain seemed a good fit. Scientists do research and peer review each other in exchange for a token. Other scientists can then easily access and improve/build on existing science, with no journal incentives required to keep academics afloat. And yes I am aware safeguards will need to be put in place to prevent gaming the system, but using AI, it should be relatively easy to actually capture the entire history of science on the chain and verify the knowledge.

If you read the whole article, that's not because Rafael Luque was using ChatGPT to churn out papers, but because he was lending his name to research papers done in Saudi Arabia and Chinese universities.

> that's not because Rafael Luque was using ChatGPT to churn out papers, but because he was lending his name to research papers done in Saudi Arabia and Chinese universities

...who were using even worse models to churn out papers about "vegetative electron microscopy."

I am aware of that and did not state otherwise, I just had to point it out. I even said "even if this is unrelated to the story"...

> Without me, the University of Córdoba will drop 300 places in the Shanghai ranking. They have shot themselves in the foot

While that was the issue that the university took with him, the extra logos on his lab coat, why does the university care about him? Why is he valuable?

Numbers. He boosts numbers with bullshit. Numbers that are misinterpreted to indicate value.

And he’s gotten good at bullshit. He’s even incorporated ChatGPT into his workflow.

So good at it that entities paid him to try and sneak some extra logos on his lab coat. He’s smart. He likely found ways to never accept money and yet fully utilize it for his benefit.

This all matters.

The system is fractally broken.

An equivalent would be to claim that you work for Apple when you have a full time contract with Microsoft.

On the other hand this game has been played before. Everybody knows in Spanish speaking science that just adding one researcher from the Anglosphere with a nice English name to the list of authors is a seal of approval for many journals, and will open a lot of doors even if the researcher just agreed to sign on the paper.

It depends on how he is using it. If he is using it to insert new content into his papers it is concerning. If he is using it to reword his papers but the content remains the same it's less concerning.

It seems to be more than that:

> Magazinov mentioned that a non-existent “vegetative electron microscopy” appears in two studies by Luque published with Iranian colleagues.

Is it unreasonable to surmise that vegetative electrons are very small, thus the microscopy?

The decision to turn off life support for vegetative electrons was made at the first Solvay Conference in 1911, at which point they ceased to exist in our universe in a total wave function collapse. ChatGPT has apparently overfitted on late 19th century theoretical physics journals.

But there’s speculation that vegetative electrons may still exist in the brains of the people who have been peer reviewing these papers.

Well there are Electron Microscope studies of the vegetative Cellular Life, so that is not the knife he will fall on...

| If he is using it to reword his papers but the content remains the same it's less concerning.

How is that less concerning? Rewording conclusions or the abstract, even subtly, can change the meaning of those words and of those sections.

I don't see how this would be much different from a non-native English speaker getting a colleague to help with the English phrasing of a paper (this is very, very common).

As long as the original researcher reads what's been written, and agrees with it, and the output gives an accurate description of the procedure and results, there's no harm done that I can see.

Edit: I mean in general, not necessarily in this specific case. This guy seems a little...questionable...for other reasons.

> I don't see how this would be much different from a non-native English speaker getting a colleague to help with the English phrasing of a paper (this is very, very common).

It's as different as ChatGPT is from an English-speaking scientist colleague.

For example, a colleague will probably ask if they're not sure which is the intended meaning of a phrase. ChatGPT will generate one of the possible meanings.

That sounds like gaming the system. I have a very hard time imagining somebody can produce quality work every 37 hours non-stop.

Well, work of SOME type of quality, perhaps.

The most prolific mathematician in history, Paul Erdős, published about 1500 papers over about 60 years. That comes out to a little more than 2 a month.

That's close to an upper bound on what should be possible without some form of cheating.

Erdős took amphetamines, so he was cheating too.

He also co-authored most of his papers and offloaded a lot of the writing to others.

But he really did play a significant role in getting the results in every one of his papers.

Mostly is publishing by being co-author: https://orcid.org/0000-0003-4190-1916

Prolificness aside, what has he actually done? Is he advancing chemistry or just co-signing dubious papers?

Co-signing dubious papers

From the article, this corruption scheme goes way beyond cosigning papers. The main selling point of this corruption scheme was gaming institutional metrics.

From the article, the Spanish researcher gamed publication metrics by co-signing huge volumes of research papers, and proceeded to sell his affiliation to the highest bidder as it allowed low-tier institutions to game rankings and bump up their standing.

There's also another angle to his scheme, as his threat to the university of Cordoba consisted of "you har my interests and your university will drop in rankings".

Seems like he was gaming the various academic ratings systems by getting lots of things published with his name on them.

I found this part quite impressive until I read the following part:

If that first stat didn't pin your bullshit detectors into the red, you need new bullshit detectors.

> Even if this is unrelated to the actual story, I find it a tiny bit disturbing. Is this the new standard practice? Let some robot spew out paragraphs of text to convey the scientist research?

I wouldn't be surprised if that statement wasn't a bold face lie, and Raphael Luque was just stapling his name to papers and proceeded with a clickfarm-like business model, where he sells the impact that his publication metrics has on institutions who request his services.

The defiant tone in his reply is evocative of other corruption cases where the criminal is so confident in his clever scheme that he even taunts defiantly everyone to challenge it. Even the old "I don't have a cent in my bank account" bullshit excuse is as old as time. Next he might just say that he only has good, generous friends who give him some presents from time to time.

I'll go one step further than the sibling. For every possible occupation, if chatGPT can improve your odds of success (except maybe for writing fiction, but I'd guess it's bad at this too), then there is something fundamentally wrong with that occupation.

This is nonsense. You might as well say "if literacy can improve your odds of success, then there is something fundamentally wrong with that occupation".

GPT is pretty reliable at taking an existing body of text that fits entirely in a prompt and doing something with it--reword, translate, answer questions about it, so forth. Even expanding an outline into written sentences usually works great as long as the outline has all the detail.

It's really only when you let it introduce "facts" based on its body of training that it'll start going off the rails. If he's just running text through without checking the final results, that's really not wise, but the LLM is unlikely to start introducing novel facts or changing figures in response to a conservative request based entirely on the prompt text.

Using it as a writing aid the way he claims he is (not that I necessarily believe all his claims) is probably one of the more responsibles way to use GPT, honestly, because you have full opportunity and ability to vet the output. Hack together something that has all the right content, then make it readable with the AI engine the way you might with a human editor collaborating, while jointly making sure the editor doesn't change the actual facts of the content.

Whether or not he could be hacking anything of value together every 37 hours is another story.

If the domain expert is there to audit the results, where is the issue?

I can't tell whether this is a question from someone who didn't read the article, or whether this is the most meta comment I've ever read.

It could be both at the same time :-)

You're assuming they are going to do as thorough of a job auditing as they'd do on their own writing. I doubt that's what will happen when the point of the AI is to spew content out faster.

This person is already writing an article in 2-3 days. Is ChatGPT realy the problem, for reducing the time to 1 day?

I think you have a good point, but you're probably getting downvoted because on first read it came off more as you were defending the use of chatgpt (rather than criticizing the absurdly fast output).

Please correct me if I'm wrong, but you're pointing out the absurdity of publishing every 2 to 3 days and implying that it was already essentiallly garbage he was producing, and chatgpt was just polishing the garbage (which IMHO, as you stated, the garbage production is clearly the bigger problem, not the polishing).

> you're pointing out the absurdity of publishing every 2 to 3 days and implying that it was already essentiallly garbage he was producing

Yes, exactly.

I'm doing this too on a paragraph level. It is amazing how much better my writing is getting with a tutor.

It is entirely unsurprising that a non-native writer would do this. It would be more surprising in five years that anyone is not doing this.

most papers are made of 90% of useless fluff, and 10% of meat.