HN commenters split over whether quitting is a moral obligation
2 Sep 21 12:00 PM · 4d ago · 2 posts · 12 comments · 2 sources · development 2 of 2
Commenters debated whether individual departures like Coxon's meaningfully address systemic AI risk, whether 'good' and 'bad' can be cleanly defined for tech work, and whether the essay's author is consistent given a stated skepticism about frontier AI capability elsewhere in the blog.
“I'm not sure what more we need to wait for, if the claim "10% it ends humanity" is not enough.”
ArcHoundJacob Coxon Former Anthropic pretraining researcherEvan Hubinger Anthropic alignment team lead
Frances Haugen Former Facebook employee, 2021 whistleblower
The whole story articlespostscomments the bright band is this development · numbered dots are the others · click one to jump
What people said 19 voices · verbatim
-
I would rather say the argument is simplified, but on a decent track. Because there are two angles to it.I've been wondering what sectors I wouldn't maintain and develop infrastructure for, because I consider it a strength of mine that I don't entirely care about that. I'm not excited about your product, for good or bad.Defense for example is on…
-
A
Trading morale for money is not a career decision but a transaction. It is a transaction where you sell the part of you that was supposed to care. Do that often enough, and it does not come back. That is not burnout, but it is still a disease. Although, like every disease, it spreads: first to your team, then to your product, then to the people…
-
I disagree with this article and here's why.Life isn't black and white with "good/bad". Further, there's no objective measure of what is good or bad. Everything is a shade of gray.You certainly could argue that some actions are absolutely good or absolutely bad, but those aren't the majority of actions people take on a day to day basis. People…
-
M
Claim of the moment: "good people refuse to do bad things" https:// carette.xyz/posts/good_people_ refuse_to_do_bad_things/ # MachineSociety
-
I like to think of good in terms of direct positive effect, and bad in terms of direct negative externalities. Second order effects become too fuzzy.Do the people working for AI labs believe that it has a better direct beneficial effect, or makes for a worse outcome on society?What I've read thus far heavily lead me to believe that the balance is…
-
Nice to see people standing up for their beliefs even when it could hurt them.Story time :)I remember when I was first starting out as a programmer. At QE (always occurred on a weekend) a slew of financial reports were created and printed for review the following Monday then I think used for summaries for Wall Street to look at.A high level…
-
There may not be a definition of “good”.But there are several workable contextual definitions of bad; it’s your job to know what they are.For example:- it would cause a harm to someone who was not made aware of the risk up front.- it would cause someone to lose a job they didn’t know was under threat.- the way the harm occurs precludes its only…
-
>The accusation>So, why are you still working for them?>If the answer is the salary, say it louder, and do not force yourself to make excuses.>If the answer is “someone worse would replace me”, say it too, and louder too.>If the answer is “I can change things from the inside”, then show me what you changed. Haugen tried the inside. Coxon tried the…
-
Generally, I agree.Imagine: you start working at the company with a good will towards them. As you learn more and see more decisions, you slowly realize how bad it is. Now you've already contributed. Then you leave.It might be possible to build everything with this goodwill and that's how we'd still end up with a torment nexus even if you…
-
> More than half of tech workers (53%) would steer a newcomer away from their own field, and they stay.It would be nice to see this survey repeated across professions. In my experience, most people have gripes about their profession because it’s what they know. Only half of respondents saying they would steer newcomers away feels a lot better than…
-
I understand the concern about your work being used to cause harm, but I also worry about the development of 'forbidden knowledge'. What does it mean to say a branch of science and engineering is too dangerous of a path to go down? Does it mean scientists are persecuted just for asking questions and designing experiments to find answers? Will…
-
A few big good/bad decisions are more important than thousands of trivial decisions.Balancing good against bad is not clear, I agree, but that makes it all the more important to think about it when making a big decision such as what job to take.Its hard and people do not even agree about how to balance it - e.g. whether the ends justify the means.
-
> do not consider yourself a contributor to humankindGood lord. You could apply the same logic to any taxpaying resident of any country whose government is doing something bad. There's some truth to it, it's worth bearing in mind, but it's also the very definition of an Isolated Demand for Rigor.
-
The author of this blog is an AI skeptic (see the backlog of posts). That's fine, but it's hard to square with this particular post. To take Coxon's argument seriously, you have to believe frontier models are hypercapable, and not just hype objects. That's what Coxon believes and why he left.
-
If everyone doing a 'bad thing' stepped off the hamster wheel today, the world economy would collapse before markets closed. The biggest companies have built an attention and surveillance nightmare machine that funnels all the money to a relatively small number of shareholders.
-
I agree.In the particular case of AI companies, I think a handbrake should be pulled when they start claiming things like "10% chance we're ending humanity."
-
You can also use moral relativism to cope with doing basically anything unethical, especially so when you’re offered dump trucks full of money.
-
How convenient would it be if the line between good and evil cuts between individuals and not through every human heart.
-
It's refreshing to see a well-argued and balanced essay on AI risk being upvoted on Hacker News.
All 2 developments of Ex-Anthropic researcher Jacob Coxon's resignation letter… →
Hacker NewsMastodonNewswires