Seems like a mostly reasonable list of things to not let AI do without better safety evals.
> AI that tries to infer people’s emotions at work or school
I wonder how broadly this will be construed. For example if an agent uses CoT and they needs emotional state as part of that, can it be used in a work or school setting at all?
This quote is inaccurate. The actual wording is: "the placing on the market, the putting into service for this specific purpose, or the use of AI systems to infer emotions of a natural person in the areas of workplace and education institutions, except where the use of the AI system is intended to be put in place or into the market for medical or safety reasons;" and it links to https://artificialintelligenceact.eu/recital/44/ for rationale.
So, this targets the use case of a third party using AI to detect the emotional state of a person.
We need to profile your every thought and emotion. Don't worry though, it's for medical or safety reasons only. Same for your internet history... You know, terrorism and all. Can't have that.
I once tried building a Macbook app which lowers my music volume when I'm not smiling, to try to stimulate more happy emotions (forcing yourself to smile induces happy feelings)
Then I started thinking how this could be used in restaurants to see if waiters smile to the people they are serving
Or in customer service (you can actually hear it when people smile on the phone)
Then I realised that this kind of tech would definitely lead to abuse
(btw that's not the reason I didn't build it, it was just not that easy to build)
We cannot extend your work contract, based on your performance the last week. You definitely did not smile enough. Our systems indicate, that you were also not happy enough a sufficient portion of the time you worked.
This is an interesting idea (the first one..) and I suspect it'd be quite easy to build now. Someone posted a web app recently that reminds you to blink (iff you don't), and it was incredibly accurate for me, running fully in the browser.
> Seems like a mostly reasonable list of things to not let AI do without better safety evals.
Yes. This is how you know that all the people screaming about the EU overregulating and how the EU will miss all that AI innovation haven't even bothered to Google or ask their preferred LLM about the legislation. It's mostly just common sense to avoid EU citizens having their rights or lives decided by blackbox algorithms nobody can explain, be it in a Post Office (UK) scandal style, or US healthcare style.
The EU generally (so far) has passed reasonable legislation about these things. I'd be surprised if it was taken more broadly than the point where a reasonable person would feel comfortable with it.
I would imagine that such a tool to infer emotional states would be most useful for autistic people who are as I can attest, somewhat handicapped upon that front. Maybe that will get challenged as disability discrimination by some Autistic group. Which would be interesting. As with most things, there are rules, and exceptions to those rules - no shoe fits everyone, though forcing people to wear the wrong shoe size, can do more harm than good.
> I would imagine that such a tool to infer emotional states would be most useful for autistic people who are as I can attest, somewhat handicapped upon that front.
It might well be a useful tool to point at yourself.
It's an entirely inappropriate one to point at someone else. If you can't imagine having someone estimate your emotional state (usually incorrectly), and use that as a basis to disregard your opinion, you've lived a very different life to mine. Don't let them hide behind "the AI agreed with my assessment".
On the other hand, as someone who's emotional state is routinely incorrectly assessed by people, I can't imagine a worse hell than having that misassessment codified into an ai that I am required to interact with.
> I would imagine that such a tool to infer emotional states would be most useful for autistic people who are as I can attest, somewhat handicapped upon that front.
The regulation explicitly provides an exception for medical reasons:
Article 5:
1. The following AI practices shall be prohibited:
[...]
(f) the placing on the market, the putting into service for this specific purpose, or the use of AI systems to infer emotions of a natural person in the areas of workplace and education institutions, except where the use of the AI system is intended to be put in place or into the market for medical or safety reasons;
I can definitely find you autistic people who would hate having such a device pointed at them, because they don't mask the ""correct"" emotional state well enough.
Was on about Autistic people being able to use it to understand the other person, not the other way around, which would be a nightmare for them, which sadly is what they get in real life anyhow already. Though might be useful to warn them that they are sending the wrong signals, so a mixed bag on that way around.
A difficulty I have with customer service folk is that usually I'm just trying to report a bug. I'm not upset. Please stop trying to give me coupons. I'm not trying to cancel my account I just want to help your engineers fix this bug (and later on, I want to see that it has actually gone away).
If I must intact with an AI for this, I'd prefer that it infer my emotions correctly.
> AI that tries to infer people’s emotions at work or school
I wonder how broadly this will be construed. For example if an agent uses CoT and they needs emotional state as part of that, can it be used in a work or school setting at all?