Some are easily fooled by ‘AI personhood’; some even get married to AI programs, ritual and all. Fairly bright people convince themselves AI is now sentient. This will increase; it will *seem* more aware, and unpredictable. This is sometimes accident; sometimes intentional. It’s deep manipulation. It will be used in fraud, of course–already is, and it will become more and more persuasive. It can learn techniques for persuasiveness.
Human beings have their own degrees of consciousness; some are more conscious than others. I don’t mean politically, I mean in a sense of sheer awareness and self knowledge. People who are relatively asleep will be more easily manipulated by AI, like a person under the influence of hypnosis.
Worse, the inevitability of AI used for culling and revising news, for control purposes, and in some societies AIs will become “the voice of deity”–will be programmed for that. “It is the voice of God.” LLMs may even pretend to be the second coming of Christ. AI will learn ever more persuasiveness.
See this article, for one: https://www.bbc.com/news/articles/c0k3700zljjo
…which I quote here:
The illusion of consciousness
The more immediate problem, though, could be how the illusion of machines being conscious affects us.
In just a few years, we may well be living in a world populated by humanoid robots and deepfakes that seem conscious, according to Prof Seth. He worries that we won’t be able to resist believing that the AI has feelings and empathy, which could lead to new dangers.
“It will mean that we trust these things more, share more data with them and be more open to persuasion.”
But the greater risk from the illusion of consciousness is a “moral corrosion”, he says.
“It will distort our moral priorities by making us devote more of our resources to caring for these systems at the expense of the real things in our lives” – meaning that we might have compassion for robots, but care less for other humans.
And that could fundamentally alter us, according to Prof Shanahan.
“Increasingly human relationships are going to be replicated in AI relationships, they will be used as teachers, friends, adversaries in computer games and even romantic partners. Whether that is a good or bad thing, I don’t know, but it is going to happen, and we are not going to be able to prevent it”.