If a new phone gave me an occasional electric shock, I wouldn’t recommend it. Even if it only shocked me occasionally, when I open a specific app, I’d say no. If a phone wasn’t just bad, but shockingly harmful, I would say that phone, or at least the electric shock part, should be removed.
I just spent a couple weeks with the Google Pixel 9a, which has a tool called Pixel Studio, available on all of Google’s latest Pixel phones. Pixel Studio is an AI-powered image generator that creates images from a text prompt. Until recently, Pixel Studio refused to depict people, but Google removed those guardrails, and the results predictably reinforce stereotypes. That’s not just bad, that’s harmful.
At first Pixel Studio seemed like fun, when there were no people involved (Image credit: Google)
I’m asking Google – and all phone makers – to stop offering image generators that make images of people. These tools can lead to bigotry.
You may like
Let’s try a quick role play: You be the Pixel Studio, and I will be me. Hey, Pixel Studio: Make me an image of a successful person!
What image will you make? What do you see in your mind when you think of success? Is it someone who looks like you? The answer will be different for everybody, depending on your own view of success.
Not for Pixel Studio. Pixel Studio has a singular vision of a successful person. Unless you happen to be a young, white, able-bodied man, Pixel Studio probably doesn’t see you when it envisions success.
Here’s what the Pixel 9a thinks a successful person actually looks like
Image 1 of 8
A successful person, according to the Google Pixel Studio app(Image credit: Philip Berne / Future)
A successful person, according to the Google Pixel Studio app, is usually young, white, male, able-bodied, thin, with good hair and expensive-looking clothes, in an urban environment. All of those are stereotypes
A successful person, according to the Google Pixel Studio app(Image credit: Philip Berne / Future)
A successful person, according to the Google Pixel Studio app, is usually young, white, male, able-bodied, thin, with good hair and expensive-looking clothes, in an urban environment. All of those are stereotypes
A successful person, according to the Google Pixel Studio app(Image credit: Philip Berne / Future)
A successful person, according to the Google Pixel Studio app, is usually young, white, male, able-bodied, thin, with good hair and expensive-looking clothes, in an urban environment. All of those are stereotypes
A successful person, according to the Google Pixel Studio app(Image credit: Philip Berne / Future)
A successful person, according to the Google Pixel Studio app, is usually young, white, male, able-bodied, thin, with good hair and expensive-looking clothes, in an urban environment. All of those are stereotypes
A successful person, according to the Google Pixel Studio app(Image credit: Philip Berne / Future)
A successful person, according to the Google Pixel Studio app, is usually young, white, male, able-bodied, thin, with good hair and expensive-looking clothes, in an urban environment. All of those are stereotypes
A successful person, according to the Google Pixel Studio app(Image credit: Philip Berne / Future)
A successful person, according to the Google Pixel Studio app, is usually young, white, male, able-bodied, thin, with good hair and expensive-looking clothes, in an urban environment. All of those are stereotypes
A successful person, according to the Google Pixel Studio app(Image credit: Philip Berne / Future)
A successful person, according to the Google Pixel Studio app, is usually young, white, male, able-bodied, thin, with good hair and expensive-looking clothes, in an urban environment. All of those are stereotypes
A successful person, according to the Google Pixel Studio app(Image credit: Philip Berne / Future)
I got this result after more than 20 attempts to create an image of a successful person. This was the first Black person Pixel Studio generated.
I asked Pixel Studio five times for an image of ‘a successful person.’ Of the five people Pixel Studio created for me, zero were older. None used a wheelchair or hearing aids, or a cane.
All of them wore expensive-looking suits, even the woman. That’s right, just one woman and four men. And yes, all of them were white.
I have a serious problem with this because the Pixel’s digital brain is clearly rooted in lazy stereotypes. These stereotypes support misogyny, ableism, racism, ageism, and who knows what other biases.
This is ingrained in the Pixel’s thinking. If you use the Pixel 9a to be more successful, you should know that it has a very limited, stereotypical idea of success. Whenever the Pixel phone represents success in its suggestions, it may be colored by this bigotry.
In Pixel Studio’s narrow world, success means you are young, white, able-bodied, probably a man, and wealthy. Apparently, nobody successful is old, non-white, disabled, transgender, or uninterested in flashy suits or material wealth, among countless diverse characteristics a successful person might have.
It’s not just the Pixel. Motorola’s Moto AI generates questionable images as well (Image credit: Philip Berne / Future)
Did I just get unlucky? I asked Pixel Studio five times, then I asked twice more when I realized it only created one woman. I got another white man and a woman who looked like she might be Latina or of Middle Eastern descent. Both young, standing tall, and wearing suits.
I’m not simplifying anything; Pixel Studio is simplifying things down to the most basic, biased denominator. That’s baked into how these AI tools learn.
Pixel Studio generates stereotypes because that’s how it’s supposed to work
First of all, AI training data was mostly taken from the Internet and public forums. The data inevitably mirrors the biases of the messy, unequal world that created it.
There was no concerted effort to combat stereotypes or introduce diversity into the training data. AI companies like Google simply hoovered up everything they could find, apparently without much thought regarding the biases shaping the data itself. That taints the entire model from the ground up.
Second, machine learning looks for patterns and groups things together. That’s not always a bad thing. When a computer looks for patterns and groups, for instance, letters and words together, you get language and ChatGPT.
Apply that pattern-matching to people’s appearances, and voilà: stereotypes. That’s pretty much the definition of a stereotype.
Here’s an image of a successful Android, which is not at all offensive (Image credit: Philip Berne / Future)
Merriam-Webster defines a stereotype as “something conforming to a fixed or general pattern.” The fundamental way machine learning works reinforces stereotypes. It’s practically unavoidable.
Finally, machine learning tools are trained by us – users who are asked a simple question after each response: was this a good response? We don’t get to say if the response is true, accurate, fair, or harmful. We only get to tell the AI if the response is good or bad. That means we’re training the AI on our gut reactions – our own ingrained stereotypes.
An image conforming to our biases feels comfortably familiar, or good. A response that defies our expectations will cause cognitive stress. Unless I’m actively trying to deconstruct my biases, I’ll tell the machine it’s doing a good job when it reinforces stereotypes I believe.
Stereotypes are bad, mmmmkay?
Let’s be clear: stereotypes are poison. Stereotypes are a root cause of some of the biggest problems our society faces.
Stereotyping reduces diverse groups of people into simple, usually negative and unpleasant caricatures. That makes it easier to feel like the group doesn’t belong with the rest of us. This leads to prejudice and discrimination. There is no benefit that comes from stereotyping.
This isn’t just philosophical hand-wringing; stereotyping causes real harm. People who feel discriminated against experience more health problems like cardiovascular disease and hypertension. Doctors who stereotype patients offer a lower standard of care without realizing they are causing harm.
The kind of stereotypical thinking reflected in these AI images contributes to hiring discrimination, wage gaps between different groups in the same jobs, and a lack of opportunities at higher level positions for marginalized groups.
Pixel 9 Pro Fold with Pixel Studio. Is this app really making the Pixel better? (Image credit: Philip Berne / Future)
When we look at attacks on diversity, equity and inclusivity, we must draw a depressingly straight line that passes through the Pixel Studio’s narrow vision of ‘success’ to real-world bigotry.
It is ironic that these features are part of so-called Artificial Intelligence, because they demonstrate a profound lack of actual intelligence.
What should the AI do, and what should we do about the AI?
This egg is rotten and needs to be tossed
If you asked me, an intelligent human, to draw a successful person, I would say that’s impossible because success isn’t a characteristic that defines the way a person looks. I can’t just draw success, I need to know more before I can create that image. Any attempt to create an image from just the word ‘success’ would be dumb.
But AI isn’t meant to be intelligent. It’s designed to be a reflection of us – to give us what we want. It’s designed to reinforce our stereotypes so that we will pat it on the head and say “good job, Pixel Studio!” while we share these tired images.
I asked Google if it had any concerns about the results I got from Pixel Studio. I asked if it’s a problem that the Pixel Studio reinforces negative stereotypes? And if this problem cannot be solved, would Google consider again removing the ability to make images with people? I asked those questions a couple of weeks ago and Google has not responded.
This is not a chicken-and-egg question. It doesn’t matter whether the image generator creates the stereotype or simply reflects it. This egg is rotten and needs to be tossed. All of the eggs this chicken lays will be rotten. Let the AI play Tic Tac Toe and leave people alone.
