back
18 comments
As smart as the people are at OpenAI they sure do some incredibly stupid things.

I guess it proves that this "company" is ran by scientists and not business people, which I guess is reassuring?

The company is run, somehow, by people who have never run much of anything in their lives. Results are as expected.

How that situation came about would be an interesting story in itself.

Does that not apply also to Sam Altman?
When he got started, yeah. I don't know how he became a YC partner without investing a lot more than he would've netted from the Loopt exit.

At this point I don't think anyone can question his track record, based on OpenAI's success at both R&D and fundraising. They are apparently trying to get him back onboard, if that means anything...

I mean most of the people high up in the company as well as the entire board appear to be fairly junior (early to mid 30s).
Did only Sam have Satya Nadella's phone number? Seems like a rude surprise for a Friday afternoon.
Your concern for Microsoft feelings is noted...
OpenAI has been bleeding money for a while [0]. Why would it be so crazy that the board wants heads to roll? The craziest part really is that he literally was allowed to give a keynote like a week ago (a good one), but of course, have they sacked him before... what kind of message that sends. Instead, he got sacked after, ANNOUNCEMENTS WERE MADE!

0: https://futurism.com/the-byte/openai-losing-money-chatgpt

1. It seems like everyone in the space is loosing money and yet investors still seem willing to just keep throwing more at them.

2. From other accounts, the people they fired were the ones leaving more heavily towards commercialization. Based on what I’ve read (which is certainly multiple steps removed from the source), these would have been the wrong members to oust if the goal was to make more money.

This really feel like a supervillain origin story though

Now Sam should make a green goblin mask and learn how to fly around town

Ilya is spiderman then lol.
If this was a division over the safety of models, the safest thing they could do is to release older models as soon as they release a new one.

If they release GPT-5 (closed), then they should release GPT-3.5 with its weights, stack, and documentation.

Real research can be done to see what safety precautions can be learned from it, as the successors are quite similar. For example, GPT-3.5 is still updated with features as of Dev Day, and gives a quite similar tech stack to the newest models.

--> What no one on Earth wants to bet on is a few people in San Francisco figuring safety out, when the rest of the world could be working on that. <--

If a model is leaked to the public, you have no time to prepare for safety. The damage is done.

If a model is kept forever behind closed doors you do not follow your mission of safe AGI. This only proves that overnight moves could be made that destabilizes who has control over the most powerful models. At least an open release of older models shows your dedication to innovation and safety.

I'd bet that since Microsoft has a copy of the stack and weights, and that the US govt has their own Azure instance cloud, the US already has a copy of OpenAI's technology. So its not going to die if OpenAI crumbles. It just goes to two different hands.

---

And this is just a bare minimum step. Not even considering this means there is a larger problem than ideological division. It is more like a power grab.

*As much as people think that releasing models is a safety risk, it might just be the best thing you can do.*

---

Economical Steps for a release that satisfies safety and investment (don't give away your MOAT):

Current Model: GPT-4; Faster/Cheaper option: GPT-3.5; Fully Release GPT-3

Current Model: GPT-5; Faster/Cheaper option: GPT-4; Fully Release GPT-3.5

Current Model: GPT-6; Faster/Cheaper option: GPT-5; Fully Release GPT-4

Define "safety," because for me and others, the definition seems to be "not projecting politically-incorrect views or information" or just straight up censorship.
At bare minimum it is to define and understand what are the capabilities of a model.

Just like with missiles. They could be short range, ICBM, or nuclear.

So if a model could make real looking videos that could confuse even expert analysis, and the general public, then we should prepare by doing X, Y, Z, so that the public cannot be confused by fake video, since we have some preventions.

X, Y, Z, are mitigations. They do not, and will not work if they are censorship, since the other person can release a model that is uncensored and use it. That misses the point of the problem. Since others have it, your mitigation is to make software systems where you can sort posts by being real or fake. How you do it is an example of safety research.

Safety is not talking about restricting models, it is talking about mitigating the negative effects of a weapon used the wrong way. Same thing with code. Malware will exist, so you must make a firewall, or train employees to not click phishing links, ...

The safety research is finding the best methods of mitigation, and a small team in SF is not going to be the best at doing this. If the world knew the extent of a model, they know what extent of mitigations they can do.

Overnight actions and power struggles in an organization can happen. If the world had these models, even the risk of OpenAI using them in a bad way, or restricting access to just themselves on usage of models, would be a more minimal risk issue, since two models ago are open.

"Safety" = "Our company is safe from competition, thanks to captive regulators."
The only thing that immediately comes to mind is you don't want a model capable of writing a Stuxnet-type cyberweapon for a script kiddie to use.

All the other stuff about nuclear/chemical/biological/illegal drugs etc. is basically nonsense, that information is in the public domain already and the way governments try to control it is through legal restrictions like Select Agent Rules. Likewise, the result of giving most people a recipe for a chemical weapon is that they'll end up poisoning themselves, and those who won't already possess the requisite skills (happily the vast majority of trained experts in these fields aren't psychopaths). The real risk here is governments doing such things in the name of national security, I think.

The other argument that many have adopted, that LLMs will make such cunningly clever arguments that gullible humans will follow them to their own destruction (I assume this means getting the masses to embrace communism or fascism or tax the uber-wealthy or something like that) seems utterly ridiculous to me, even though that's probably the "AI threat" I've seen the most ink spilled about. The best defense against such dark arts is simply teaching critical thinking skills to children.

So why the spectacle? Why the melodramatic press release and thoughtless communication with investors?

I will not shed a tear for Altman or Microsoft, but this fantastic Friday massacre, don't seem to line up with cause.