Digital scammers have never been so successful. Last year Americans lost $16.6 billion to online crimes, with almost 200,000 people reporting scams like phishing and spoofing to the FBI. More than $470 million was stolen in scams that started with a text message last year, according to the Federal Trade Commission. And as the biggest mobile operating system maker in the world, Google has been scrambling to do something, building out tools to warn consumers about potential scams.
Ahead of Google’s Android 16 launch next week, the company said on Tuesday that it is expanding its recently launched AI flagging feature for the Google Messages app, known as Scam Detection, to provide alerts on potentially nefarious messages like possible crypto scams, financial impersonation, gift card and prize scams, technical support scams, and more. Combined with other AI security features for Google Messages—all of which run locally on users’ devices and do not share data or message content with the company—Android is now detecting roughly 2 billion suspicious messages a month.
“The fraud is truly heartbreaking,” says Dave Kleidermacher, vice president of engineering at Android’s security and privacy division. “There’s really a very huge amount—almost epidemic and a scourge to humanity—of financial scams that are all across the world.”
Scammers operate all over the world, but Chinese scam groups particularly are behind millions of fraudulent messages, demanding things like “toll” payments or information for alleged postal service deliveries. When people click the links and enter their details, including payment information, scammers steal their data. In some cases, the scams are designed as a sort of smash-and-grab, where attackers quickly trick users into giving up some crumbs of information, like a pair of login credentials or a credit card number. These scams tend to be more formulaic and are potentially easier to detect. The more complex challenge is in detecting highly involved investment or romance scams—often called pig butchering scams—that build and evolve over months of messaging while scammers build a rapport with their targets before tricking them into handing over their life savings or even going into debt to send more money.
“It takes time for them to get to the scam—it’s not just click on the link,” Kleidermacher says. “By having the AI on-device, you can actually watch and observe these more sophisticated conversations and then detect their scams.”
Courtesy of Google
Courtesy of Google
In a screenshot of the Scam Detection feature provided by Google, an encrypted RSC chat shows a typical scam message saying an EZ Pass toll payment is outstanding. The message adds that the “legal ability” to drive may be revoked if the payment is not made. The message includes a link that directs someone toward a malicious payment website. The Scam Detection overlay at the bottom of the screen says that “suspicious activity” has been detected in the message and offers a way to report and block the sender, alongside an option that allows people to flag that it is not a scam.
Google is far from the only company using AI to try to combat scammers and stop them from reaching people’s inboxes. Some have turned to using AI to directly fight back against scammers. The British telecom company O2, for example, created an “AI Granny” that is set up to keep scammers on the phone and waste their time. And the online scam baiter Kitboga has created a series of bots to make simultaneous calls to call centers that run scams.
Meanwhile, in recent months, Meta, which owns WhatsApp, Messenger, and Instagram, has started to introduce pop-up warnings when people are asked to make payments in chat messages. Elsewhere, cybersecurity company F-Secure has created a beta tool to help people identify if a message and sender are likely scammers and block messages. Putting a layer of friction in place that nudges people away from messaging accounts they don’t know or replying to messages asking for details can reduce the chances that scammers are successful.
Google’s Kleidermacher says that the company is seeing “really positive impact” from using its machine learning systems to detect potential scam messages in real time. As the protections continue to mature, he notes that the underlying system could eventually proliferate beyond just the Google Messages app into third-party communication platforms.
For now, some of that expansion is starting within Google’s own products. The company also said on Tuesday that it is in the early phases of testing ways to incorporate scam detection for phone calls, but the capability has not been widely deployed.
