Item — HN Reader

In VC investing there is the saying that most of the value comes from the outliers. I think it’s the same for a lot of domains. I read an interesting article recently on LLMs homogeneity when writing fiction: https://arxiv.org/abs/2604.03136

It seem that at scale LLMs output is the average of their dataset, they all cluster around the same space, where human creativity comes with more variance, exploring way more of the space

2 comments

Can't the LLMs build scripts/tools etc that help generate creative ideas?

e.g. have the LLM generate multiple lists of characters, themes etc and then have a script hooked up to a RNG pick from the lists to create more "creative" ideas.

I think there’s merit to this approach, particularly to highly parallelizable tasks.

Rather than giving many agents the same prompt, introduce random variations that lead each agent in different directions. For a single bug, you might fire three agents, and later select the best result:

“Fix this bug. The solution is a trivial typo.”

“Fix this bug. The solution centers on correcting a bad assumption.”

“Fix this bug. The solution will require a complete redesign.”

You could follow the same idea with varying the input context, or by adding artificial constraints to the solution. Like telling each agent to “fix the bug, by only modifying file a/b/c”

That's pretty low-value creativity. Real innovation doesn't come from mad libs.

I would highly recommend the "Everything is a Remix" series on YouTube to show how a lot of "creative" work is really just combining different items.

https://www.youtube.com/watch?v=nJPERZDfyWc

I fear there is a strong sentiment that people are somehow special and magical and AI will always be a pale comparison.

The truth is likely that people are simple heuristical machines. There are already studies that strongly suggest our consciousness is a story our brains make up and we don’t really know why we do some things.

My fear is by rejecting the idea that AIs might in-fact become as capable as people, because “people are awesome!” We will be blind to real danger. Because of our hubris.

People are indeed special, in the sense that what we consider creativity, art, etc are human concepts. They are an expression of humanness, and are evaluated against human experience and human expectations. It’s what comes from a collective human consciousness.

This is roughly the same as putting a group of random people and telling them to think outside the box which in my experiences never yields much of thinking outside of the box

> This is roughly the same as putting a group of random people

In the book Range by David Epstein [0], he mentions that research has shown that groups of people with very mixed backgrounds leads to much faster rates of problem solving and innovation. The reasoning is that someone will have an experience or piece of knowledge that directly applies to the problem.

As an example, two different groups were given the same problem to solve. One was a homogeneous group of academics. The other had a mixture of academics and non-academics. Both had to solve a problem that involved flowing liquid. The mixed group solved it faster due to one of them having had a father who was a plumber.

0 - https://amzn.to/4wWd50y

i mean i agree diversity of opinion is good for problem solving, but the comment you were replying to is saying telling people to think outside the box doesnt really do anything. The e.g. also seems really cherry picked, im sure after a 100 samplings, the academics would solve the problem faster most of the time

> im sure after a 100 samplings, the academics would solve the problem faster

Repharase thinking:

Some problems are better for academics, other problems require other background information.

The point of the GP isn’t that they are all academics. It’s that one solution group is homogeneous and the other is not.

The claim is: the breadth of a heterogeneous groups is more likely to encompass a good idea than the depth of an academic group.

Also, I have to say that academics have (imo) little experience working together cooperatively, so even on academic problems you’d be better off with some non-academic people in there to act as buffers.

every good team I was ever on used this line of thinking during the hiring process. we whiteboard a problem we worked on in the past and discuss it with the candidate and look for anyone whose thinking about the problem raised our eyebrows!

All the LLM needs is a tiny rate of variance then you can scale it and it will out produce humans.

That’s already what we currently have. Look for “temperature” in the context of LLMs. They are already tuned to have variance.