Emily Bender is a Professor of Linguistics at the University of Washington where she is also Faculty Director of the Computational Linguistics Master of Science program, affiliate faculty in the School of Computer Science and Engineering, and affiliate faculty in the Information School.
Alex Hanna is Director of Research at the Distributed AI Research Institute and a lecturer in the School of Information at the University of California Berkeley. She has been featured in articles for the Washington Post, Financial Times, The Atlantic, and Time.
What’s the big idea?
The AI Con is an exploration of the hype around artificial intelligence, whose interests it serves, and the harm being done under this umbrella. Society has options when it comes to pushing back against AI hype, so there is still hope that we can collectively resist and prevent tech companies from mortgaging humanity’s future.
Below, co-authors Emily Bender and Alex Hanna share five key insights from their new book, The AI Con: How to Fight Big Tech’s Hype and Create the Future We Want. Listen to the audio version—read by Emily and Alex—in the Next Big Idea App.
1. The tech that’s driving the current wave of AI hype is built on a parlor trick
Chatbots like ChatGPT are impressive technology, but maybe not in the way you think. They cannot perform the range of functions they purportedly fulfill, but rather, they are designed to impress us. The key to their parlor trick lies in how people utilize language. You might think it’s a simple matter of decoding what the words say, but the process is both far more complex and far more social.
We interpret language by relying on everything we know (or guess) about the person who said the words, and whatever common ground we share with them. Then we make inferences about what they must have been trying to convey. We do this instinctively and reflexively. So, when we encounter synthetic text of the kind that comes out of ChatGPT and its ilk, we interpret it by imagining a mind behind the text, even though there is no mind there. In other words, the linguistic and social skills we wrap around AI outputs are what make it so easy for the purveyors of chatbots to fool us into perceiving chatbots as reasoning entities.
2. AI is not going to take your job, but it will make your job a lot worse
Much of the purpose of AI technology serves to remove humans from the equation at work. The story of the Writers Guild of America strike is instructive here. In 2023, the Writers Guild of America East and West (or the WGA), the labor union representing Hollywood writers, went on strike for several reasons, including a demand to raise the pay rate that writers receive from streaming services.
They also wanted to ensure that they wouldn’t be reduced to babysitters for chatbots tasked to write scripts based on harebrained ideas from movie and television producers. John Lopez, a member of the WGA’s AI working group, noted that writers could be paid the rewrite rate for dealing with AI-generated content, which is much less than the pay rate for an original script. We’ve seen the threat of image and text generators drastically reduce the number of job opportunities for graphic designers, video game artists, and journalists. This is not because these tools can adequately perform the tasks of these professionals, but they perform well enough for careers to be cut short and for workers to be rehired at a fraction of what they had been paid before, just so that they can fix the sloppy outputs of AI.
“They perform well enough for careers to be cut short and for workers to be rehired at a fraction of what they had been paid before.”
Furthermore, systems that get called ‘AI’ are often a thin veneer that hides the tried-and-true corporate strategy of outsourcing labor to people in the Majority World, also called the Global South. Many of these workers moderate online content, test chatbots for toxic outputs, and even remotely drive vehicles that are advertised as being fully automated. Luckily, workers have been able to push back, both by concerned labor action, industrial sabotage (especially through creative tools for artists, like Nightshade and Glaze, which prevent their work from being used for training image generation models), and political education.
Because we use language in just about every sphere of activity, and because the synthetic text extruding from machines can be trained to mimic language, it can seem like we are about to have technology that can provide medical diagnoses, personalized tutoring, wise decision making in the allocation of government services, legal representation, and more—all for just the cost of electricity (plus whatever the companies making the chatbots want to charge). But in all these cases, it’s not the words that matter, but the actual thought that goes into them and the relationships they help us build and maintain.
AI systems are only good for those who want to redirect funding away from social services and justify austerity measures. Meanwhile, those in power will be sure to get services from actual people, while foisting the shoddy facsimiles off on everyone else. The head of Health AI at Google, Greg Corrado, said he wouldn’t want Google’s Med-PaLM system to be part of his family’s health care journey. That didn’t stop him from bragging about how it supposedly passed a medical licensing exam. It didn’t. But more to the point, designing systems to pass multiple-choice exams about medical situations is not an effective way to build useful medical technology. In these domains, AI hype takes the form of specious claims of technological solutions to social problems, based, at best, on spurious and unfounded evaluations of the systems being sold.
4. AI is not going to kill us all, but climate change might
There was a time in Silicon Valley and Washington D.C. when an idiosyncratic, yet serious, question was posed to people working on technology or tech policy: “What is your p(doom)?” p(doom) refers to probability of doom, or the likelihood that AI would somehow kill all of humanity. This doomerism is predicated on the development of artificial general intelligence (or AGI).
AGI is poorly defined, but the basic idea is a system which can do a variety of tasks as well as or better than humans. Unfortunately, doomerism has serious purchase with some technologists and policymakers, and is predicated on a body of unseemly ideologies, including effective altruism, longtermism, and rationalism. These ideologies take the moral philosophy of utilitarianism to the extreme, suggesting that we need to discount harm in the present to save the billions of trillions of humans who will live in some undefined future. These ideologies are eugenicist in their origins and implications.
“Doomerism has serious purchase with some technologists and policymakers.”
Meanwhile, we are likely to fail to meet the Paris Agreement’s goal to limit the increase in global average temperature to well below 2 degrees Celsius above pre-industrial levels, and AI is making this problem worse. The data centers that host these tools are generating vast amounts of excess carbon, semiconductors used for their parts are leeching forever chemicals into the ground, and backup generators are projected to cause more respiratory illnesses in the poorest parts of the U.S. and elsewhere. Not only are robots not going to take over the world, but their production is going to make the climate crisis much worse.
5. None of this is inevitable
The people selling AI systems and the hype around them would like us to voluntarily give up our agency in these matters. They tell us that AI, or even AGI, is inevitable, or at least that systems like ChatGPT are “here to stay.” But none of this is inevitable.
We do have agency, both collectively and individually. Collectively, we can push for regulations that prevent AI tech from being used on us and for labor contracts that keep us in control of our work. On an individual level, we can refuse to use AI systems. We can be critical consumers of automation, being sure we understand what’s being automated, how it was evaluated, and why it’s being automated.
We can also be critical consumers of journalism about technology, looking for and supporting work that holds power to account. And finally, we can and should engage in ridicule as praxis, meaning having fun pointing out all the ways in which synthetic media extruding machines are janky and tacky.
This article originally appeared in Next Big Idea Club magazine and is reprinted with permission.
The extended deadline for Fast Company’s Brands That Matter Awards is this Friday, June 6, at 11:59 p.m. PT. Apply today.
