Well that's not a counterargument, but you're also missing the point completely, which is that you have to have a very low capacity for empathy in order to push ahead towards AGI when you know society is not prepared for this and that it's going to induce considerable pain.
Americans (let alone people elsewhere in the world) are already struggling. Recent reporting suggests a great many have to work multiple jobs. Almost all of us work an absurd number of hours per week. Many if not most can't afford homes. Plenty are ending up on the streets. Healthcare can literally bankrupt people. A vacation out of the country is an impossible luxury for most. The majority of Americans still don't send their children to college, usually because of affordability.
And I haven't even touched on what life is like in most of Africa or Asia.
This is the world we're bringing AI into. You have to be something adjacent to a sociopath to be okay with that. So long as our system is predicated on capitalism, AI may very well induce more downstream suffering than anything else humans have ever conceived.
Things aren't really that bad for most Americans, but even if they were, it doesn't follow that adding more intelligence to the world would be a bad thing for them.
A lot of people in the lower income brackets do the kind of work that an AI can't do. The people who should be worried most are actually college graduates doing clerical work, whose main work output is writing or evaluating texts. Even those people will likely use AI as a tool to enhance their productivity, because the AIs still are not good enough to replace people for tricky edge cases. The first companies that try to replace their customer support workers with an AI are going to have a bad time (and so are their customers!).
When almost everything can be automated, the problems that remain are the really hard ones that can only be solved by human experts.
A construction worker with a circular saw can cut boards way faster than someone with a handsaw -- but the introduction of circular saws didn't result in a bunch of carpenters getting laid off. Instead it made them more productive, and for people who get paid by the task rather than by the hour that is a huge benefit. They could build more and make more money, and a bunch of other people benefitted from their increased output, like homebuyers and property developers.
Similarly, as a software engineer I benefit from code generation tooling already. If that gets smarter and faster, I will be more productive, my team will be able to build software faster, and instead of laying people off I will expect to be given more work. Maybe our 4-year roadmap will be achievable in 1 or 2 years with the same size team.
Productivity gains by and large do not translate into real wage gains and an improved quality of life for laborers. We have more than a century's worth of data suggesting they usually do the opposite. Yet somehow this fairytale that productivity gains are a boon for laborers persists.
> Similarly, as a software engineer I benefit from code generation tooling already. If that gets smarter and faster, I will be more productive, my team will be able to build software faster, and instead of laying people off I will expect to be given more work. Maybe our 4-year roadmap will be achievable in 1 or 2 years with the same size team.
Why so sure the end users aren't going to be feeding their own requirements directly to a Jenkins/Copilot/ChatGPT mashup running as a service in the cloud?
People aren't "strugging" because there is no work for them to do. They are strugging because the powers that be have jiggered our economic system in order to hamper, instead of facilitate, free market cooperation, specialization, and trade. Governments micromanage everything. That is what needs to stop.
If "AI" ends up making governments think they can continue to micromanage and get away with it, yes, that will cause more suffering. But that's not the failure mode any critics of "AI" are trumpeting about.
> Americans (let alone people elsewhere in the world) are already struggling
I agree. And I agree with your overall sentiment about the risks of pursuing AGI. I'm as cynical as anyone about the likelihood that the average person will really be any happier in a world with AGI (controlled by tech billionaires no less).
That being said, to claim that hardly anyone in the tech community is capable of empathizing with the average person is a wild overstatement that brings nothing to the discussion. Just adds to the noise.
Late reply here but I wanted to point out that you still don’t get it. True empathy in the tech community would be e.g. having the courage to say that building HLAI of the kind we’re now approaching is guaranteed to cause tremendous amounts of suffering for ordinary people (who will not be able to respond elastically to so abrupt a tectonic shift), and therefore the whole enterprise is fundamentally evil.
Let’s get real concrete about what’s going to happen: people will lose their jobs, then their homes, they’ll become destitute, they’ll experience divorces, some will commit suicide, they will suffer desperately in myriad other ways due to economic disenfranchisement, kids will be deprived of a comfortable upbringing, etc.
How many in the tech industry are genuinely discussing the very real consequences of nonlinear degrees of automation for the kinds of ordinary people they barely interact with? How many are pretending that there isn’t something disgustingly immoral about having some of the most affluent and economically insulated people devise and inflict this reality upon countless millions?
I will maintain that this industry is morally bankrupt and nearly entirely devoid of empathy. These are not the people who should be in charge of our future.
> I will maintain that this industry is morally bankrupt and nearly entirely devoid of empathy. These are not the people who should be in charge of our future.
Since the tone of your characterization is so absolute, why doesn't it apply to you? Why are you here in this tech community at all if the whole industry is so morally bankrupt? Why would "present company" ever be excluded for this or that reason? Because they're your friends? You're just projecting your own anger onto an entire group of people that you mostly don't know.
What I think you mean when you say all of this is that those in control of the tech industry are morally bankrupt. And, after 10+ years of getting kicked around as an engineer, I think I would have to agree. But I'm not so foolish as to broadly dismiss everyone in the industry just like me, who started out as a silly nerd who just liked computers and math, and who is essentially still that same person at their core, as a lost cause. I don't do that because I know everyone is fighting their own fight. But it's clear that those who aren't fighting are the ones on top that are sucking the life blood out of society. I'm more and more resentful towards that demographic every year. And I agree with you that they're crossing some kind of moral line by developing this tech, or at least by trying so hard to maintain control over it.
But the tech will get developed either way. If you're in the camp that thinks we should somehow just stop doing all this, you don't seem much different to me from someone that wants to mandate encryption backdoors. Our society will never be well coordinated enough to do that correctly. This isn't like making nuclear bombs, which takes a lot of physical industry. This is something that is just months away from running on commodity gaming hardware. Probably just a few years away from running on the average laptop. It does feel a bit like a harsh reality, just like the fact that a meteor could slam into the earth at any moment. But there it is; what are you going to do about it that isn't either futile or self-destructive?
Americans (let alone people elsewhere in the world) are already struggling. Recent reporting suggests a great many have to work multiple jobs. Almost all of us work an absurd number of hours per week. Many if not most can't afford homes. Plenty are ending up on the streets. Healthcare can literally bankrupt people. A vacation out of the country is an impossible luxury for most. The majority of Americans still don't send their children to college, usually because of affordability.
And I haven't even touched on what life is like in most of Africa or Asia.
This is the world we're bringing AI into. You have to be something adjacent to a sociopath to be okay with that. So long as our system is predicated on capitalism, AI may very well induce more downstream suffering than anything else humans have ever conceived.