Item — HN Reader

> The original statement was: (1) The branch to the left from A is a dead end.

This is where you get things wrong at a very basic and fundamental level.

Just because you failed to explore branch A, that does not mean it is a dead end. It just means you came up empty.

That is why science is based on observations and theories: it is based on building up on ideas and what works and can be proven. Otherwise you will left with useless papers such as "Bicycles are a dead end because I tried to ride one and I fell".

1 comments

You are assuming incompetence on the scientist saying that the branch from the left of A is a dead end.

While nobody is perfect, there are numerous perfectly valid scientific negative results. You know, there exist things like impossibility proofs in mathematics and computer science. There are equivalents in other sciences (e.g. if X was true, that would lead to Y that is easily observable and clearly not observed). Sometimes that implication has assumptions that might change once the technology/society changes, other times it holds true regardless.

Unicycles are a dead end as a practical transportation, because the bicycles have them beat in every way (except portability).

A scientific result would be much more along the lines of 'Bicycles without gears have limited applicability, especially in hilly terrain.'

There are different types of research. Negative AI results are not like Michelson-Morley experiment in physics that can positively establish the lack of something. It's more of a craft, the "make it work" style of thing. Give the same task or goal to two different teams, you will see massively different outcomes. Simply failing to make it work is not conclusive.

To make such a negative result acceptable in AI, you'd have to have some clear reason why you think that your particular setup should produce the result you want, that exact configuration and architecture, dataset etc. There are countless projects in AI that fail. And it's not clear at all that it refutes any abstract hypothesis. It's a get-your-hands-dirty field. It can make or break a project whether someone has that tacit knowledge, that black magic experience to know how to properly do the project.

People can generate extremely many ideas. You'd need to convince me that your idea (among a million others that people are trying each day) is so significant that its failure is in itself interesting. If you were to review for AI conferences, you'd see the flood of papers that claim to achieve 0.5% or 1% improvement on some benchmark. Now imagine that they didn't even have that to show for it. It got worse by 2% after trying their random idea. Who cares then? Even the +1% with a random idea is quite annoying to accept. But if their random idea really made something work much better, I will at least have some reason to want to see what may be going on there, there can be some signal. With negative results, it's very uninteresting.

I did read OP's post much more generally than only about AI research. There are many research areas where (properly done!) negative results would be very valuable, but are not properly encouraged.

I do agree that in a new area with too many degrees of freedom (and yes, AI research is one of those), negative results (especially poorly done) are of limited usefulness.