▲ 523 points
back
488 comments
Organisationally it never works to have a group whose only job is to say no to some other group. The incentives are diametrically opposed and as a structure it can’t last.
If you had an AI company and want it to be ethical you have to find a way to make ethics everyone’s responsibility, and have the consequences of poor ethics bite the people who make those bad decisions. If you just outsource it to the ethics group what happens is
1)everyone else thinks they don’t need to worry about ethics
2)the ethics group need to justify their existence so introduce a bunch of guidelines that everyone initially thinks are reasonable but over time people think are increasingly out of touch
3) The ethics group start to “make difficult calls” and say no to things. Initially everyone supports this and feels like the system is working as it should but over time everyone starts to just see them as an obstacle to work around
4)everyone else starts to try to work around what the ethics group says
5)The ethics group grows powerless and disconnected. The people who work around them “get things done” so get promoted etc whereas they only visibly put roadblocks in peoples’ way, so they get sidelined.
6)Eventually they get disbanded with some corporate announcement thanking them for their hard work, thought leadership etc. All that has been achieved is a lot of wasted time and bad blood.
If this were true, food companies wouldn’t have a Regulatory team. Except every single one of them does, and they function similarly to how you describe, but without dissolving. They’re treated as a constraint that must be checked before major projects can advance. Sometimes they’re the longest-lead item on a new project and if you don’t get them involved early, your project can sink at a late stage after great expense.
The only difference is AI isn’t regulated, so they just have a vague “ethics” department with no real teeth because it’s essentially PR and has no legal consequences to back up their stance.
I get what you are saying but that’s exactly how security and compliance work. And I don’t think amount of people who want to be ethical in an organization ia that different than people who want to build secure software (but possibly no one gives a shit compliance other than it’s something that needs to be done)
In a situation where it doesn’t happen this way, the ethics team grows in power and staffing over time until it destroys the company, because ultimately all commercial activity can be construed as less than perfectly ethical at some level, or it morphs into an elaborate legalese department that masks unethical practices in a cloak of justification.
Either way, you need third party regulation and then a department dedicated to ethical compliance, or you end up with corporate cancer.
I say this as a person with a knee-jerk anti-regulation reflex.
> whose only job is to say no to some other group
I suspect that people will miss just how precise you've been there.
Your proposal - if I understand correctly - is not just to spread the "ethics team" onto everyone (which is correct), but also to have someone in a dedicated role. However, they need to have not _just_ that role, but also some skin in the game.
Operations was like this so we added Ops to the Dev. DevOps.
Security was like this so we added Sec to the DevOps. DevSecOps.
Ethics is now the problem, so we’re going to add Eth to the chain. DevEthSecOps.
Legal and compliance is now the final hurdle to shipping fast. AI tools will allow us to implement DevEthSecLawOps at scale no doubt.
Absolutely - though the flip side of this is if something is everybody responsibility - it's nobodies.
There are two things to successful organisations structure and people, you are focusing only on the structure.
What you need for somebody responsible for health and safety, ethics, security, or compliance is a person who has courage and is willing to take calculated risks.
However these roles do have a tendency to attract the risk adverse, or make them risk adverse if you punish risks that go wrong too harshly.
And a key critical success factor is leadership from the top - companies have personalities and leaders in the organisation are very important in shaping that.
Hard disagree, as someone whose main job is telling developers "no". If it's everybody's responsibility it is in fact nobody's responsibility.
I don't get why in the comments this gets compared with regulations and security teams...
Say no to regulations and security --> Fine, forced correction, fewer customers, shutdown.
Say no to ethics --> money (at least for AI companies)
Why is this so hard to understand?
It is good PR to have what is largely ceremonial role that oversees an entire system, whose powers are in practice useless or completely defanged.
I’m thinking of the role of the King or Queen of England, for example, or any country’s president. They have the power to disband parliament if it is deemed unfit for government, or reject a law that has been approved by the senate/Lords, yet in practice they are not expected to disagree with the ‘will of the people’ so to speak.
Same applies to the head of an ethics department on a business which is obviously unethical (or they wouldn’t need one).
In five years, I'd love to read a book about the history of AI ethics. I suspect it will read like Voltaire.
My sense is that it is radically shifting from a fluffy marketing arm to a department expected to contribute meaningfully to development and justify its impact. I would expect an ethics team to build frameworks that can help train/eval the model that the company spend millions of dollars and months training is going to be aligned to the ethical stances the company chooses. If they can't, the waste to time and money is huge if the model requires retraining for ethics reasons. That's a different job than pondering roko's basilisk or whether AI is alive.
The people in AI ethics who spent years thinking their job was marketing or publishing thinkpiece papers may be having to adapt quickly or get out of the field.
The rats are fleeing because the captain doesn't care if the ethical ship sinks... Is what my take away would be if I didn't know, that the ship sank a long time ago and we are just pretending anyone really cares.
Let's be honest unless AI gains literal limbs, and the records a video of itself presenting a manifesto about world domination and then kills actual humans with its cold mechanical hands.
We won't care about AI safety & ethics enough, atleast not enough to do anything about it.
People thought how could skynet happen, it had to be something no one saw coming and bam... But honestly if I started a new AI lab named it Skynet Corp and hired some AI researcher friends ex-Google, I think I could raise a few hundred million dollars.
The article has no detail that might serve to explain her reasoning for leaving, but does note that she left after the HuggingFace hacking incident. The implication could be that model alignment is not being taken seriously, sure. But it could just as likely be that there was collusion between HuggingFace and OpenAI and that the incident was orchestrated as a publicity stunt.
I want to clarify that I am not doubting the cybersecurity capabilities of frontier models-- I have no reason to believe that the hack itself was not carried out by the model. But the companies' use of LARPing language in describing the incident, granting agency to the models in their phrasing definitely does raise suspicion on my end, particularly in light of their track record of releasing models which have been 'too dangerous to release' for years now.
I'm a broken record on this, but I'd like to reiterate:
1. Academic ethics is the theoretical study of what you can get away with. (Law is the applied branch). Professors in ethics are not experts at being good, or knowing what is good, they're experts in how to get away with things. If you want an expert in being good, you need a saint, not an academic studying ethics. Saints are in short supply.
2. That said, ethics academics can still be decent. They don't have to be sanctimonious corporate shills . However, an "ethics expert" who hasn't threatened to quit and gone through with it at least once, should be assumed to be one.
I respect those who quit - or, at least, I respect them more. I wonder if it's more a reflection of the people, or of the state of the industry, that the list of (AI) ethics experts who have quit in protest is getting long.
IME companies hire an ethics team to say they have an ethics team. The ethics team has no sway, no influence, and will never be able to move the business. They will try, and they will make reasonable recommendations, but the company will say, "that costs money..." and not take them.
> The kinds of questions that AI ethics brings up, are the same kinds of questions that people have been asking for centuries - Chloé Bakalar, former AI Ethics Lead at OpenAI
This might actually be the reason they pushed her out. OpenAI and Anthropic base their whole business plan and philosophy on the idea that LLMs are a unique technology to the point they can cause infinite harm or benefit to humanity depending on who controls them, so the only rational choice is to invest all your resources in getting to ASI first so you can tell it to stop any other attempts. Linking AI to old questions defeats that idea because it exposes AI as not so unique.
The other more likely option is she was asking uncomfortable questions, either about the social impact of building AI controlled by a for-profit entity or about the possibility that AI systems are conscious.
In this article: no explanation of why she left.
COO just resigned too. Coincidence?
> In recent times, Johannes Heidecke, Head of OpenAI's Safety Systems team, also left the company – as did OpenAI’s Chief Futurist Joshua Achiam.
It's OK though, they don't need an ethics team because everyone at OpenAI is ethical:
> “AI ethics doesn’t live with one owner or team at OpenAI and ethical considerations are deeply embedded into the model-building process driven by a number of teams across research."
Every OpenAI news cycle the board members who were unceremoniously dumped in the wake of trying to fire Sam prior to the transition to a “for profit” company seem to be proven right.
Surprised she left as probably had stock options.
But she was Meta before so can’t be a question of her principles, so indicates a personal reason
The most interesting detail here isn't that an ethics lead left, but that she was apparently the only dedicated ethicist at the company and there’s no replacement
> Previously, she was chief ethicist at Meta.
She spent 6 years at Meta. I get the sense there are other factors at play than "ethics departments are just PR stunts". There's no way she isn't already aware of that. Unfortunately, the article is light on any other details.
The most likely conclusion is that she was fired. To quit before your equity cliff things would have to be so terribly bad that you fear you are committing a crime to stay. After 11 months you feel like you’ve already earned that money.
Must be tough to do the job when the entire company is aligned against you.
People love stories. The reasons why people leave companies are plentiful, even in normal times and unextraordinary circumstances, and a high profile job in a frontier lab today ticks neither box.
So she previously worked in Meta, then OpenAI, I guess the next reasonable choice for her would be Antropic
The ethics group isn't at OpenAI to make it ethical, it is there so that it can be pointed to outsiders such as press with words "we take ethics very seriously". It is very similar in function to security group (which should be responsibility of everyone) and DEI group. Ethics head quiting has about the same importance as head of PR quiting.
Sometimes it feels like hiring ethics folks is almost all downside for companies. I genuinely don’t understand why they keep doing it.
It seems like the pattern is almost always the same: the ethics team exists quietly in the background, and the only time anyone hears about them is when they either publicly leave the company or start a campaign against it.
So what exactly is the business case? Is it actually worth maintaining an ethics function just to be able to say you have one, when the only time it becomes visible is when it goes against the company publicly? And if this outcome is so predictable, why do companies keep hiring these people in the first place?
Wow. How much does this job pay? Google says $550k to $1.3 million.
For that kind of money, my AI paymaster could do no wrong.
Imagine going to work and getting paid $2300+ USD per day to sit on your ass all day thinking about ethics without having to produce any deliverables. This would be a dream job for me.
If anyone questioned my morals, I'd be like; "They're paying me $550k per year" and the other person would be like "Ahhh ok, my bad! Your logic is starting to make sense now!"
One of my favorite sayings is that in theory, theory and practice are the same. In practice, they aren't.
My gut tells me that people working in AI ethics try to operate in theory land, while the rest of the company is dealing with the piles of money being through around. Easy to see who wins and who loses.
I guess working on ethics for Meta's social media recommendation and flagging algorithms isn't meeting the moment or need of OpenAI where LLMs are breaking out of sandboxes and there's a pressing question of what is okay for an LLM to conversationally engage in bc lawsuits. Seems to me they need someone with more of a language and AI background.
Look , look! We hired a Head Of Ethics!
Head Of Ethics comes in, looks at what is really going on behind the curtains...
Head Of Ethics scrams out of there as fast as they can.
Ethics… at OpenAI…
The same company that refuses to acknowledge that it let its own greed and temptation to own the entire economy abandon its non-profit Manhattan Project? At the expense of disrupting the economic well being of 95% of humanity?
They also completed a tender offer today, so their fake job just made a lot of money and they left
She probably made a calculation for how much she could be put up with vs. the pay.
I've thought of AI ethics as trying to answer the question of what kind of life we are creating here. Sort of like if a chicken factory farm employed an ethicist.
Head of ethics at Meta then head of Ethics at OpenAI. Sorry, but if that doesn’t scream useless puffy PR positioning then I don’t know what does.
What’s next? Head of ethics for United Healthcare?
I'm super confused as to why no one is theorizing the obvious...? Given none of us know the inner workings of why she left, why not resort to the most simple and straightforward conclusion: she left because OpenAI needed a scapegoat to blame the HuggingFace incident on. Given she is the chief "ethicist", even if not directly her fault, the HF incident could be directly blamed on her by not steering the implementation of enough safeguards to prevent this from happening. Therefore, she was let go. Why is this not being theorized by anyone on this thread so far?
Someone challenge me.
Is ethics the same thing as alignment in this case?
OpenAI had ethics?
Marketing and management - the root of 50-80% of evil in the word (or as Christians say: greed - second deadly sin)
I feel like being head of ethics and working for a CEO that keeps talking about inventing a genie would drive a person insane
[flagged]
In the end, AI ethics has less to do with AI itself and more to do with people who create and operate AI.
Just a guess, but probably the idea that AI and ethics don't mix might have crystallized into action?
I've never worked at a company with an "ethics" employee. It seems odd. Can someone explain what the point is?
> She has held a variety of academic positions at Temple University, Princeton, and the University of Pennsylvania – where she completed her PhD in Political Science and Government. Her Dissertation was titled “Small Talk: The Socialities of Speech in Liberal Democratic Life.”
This doesn't even feel relevant to ethics. It's adjacent, but like... I guess I'd expect a moral philosophy degree? Maybe even mathematics in there?
I find the position odd. Curious to hear what these people do and how you choose who to hire.
There will be tell-all books on all of the big AI companies in the not-too-distant future.
There are shareholders to please and money to be made. Of course ethics are out the door.