This AI Writing Detector Shows Its Work. For Me, It’s a Step in the Right Direction

This article was written by an actual, flesh-and-blood human — me — but an increasing amount of the text and video content you come across online is not. It’s coming from generative AI tools, which have gotten pretty good at creating realistic-sounding text and natural-looking video. So, how do you sort out the human-made from the robotic?

The answer is more complicated than that urban legend about the overuse of em-dashes would have you believe. Lots of people write with an (over)abundance of that particular piece of punctuation, as any editor will tell you. The clues may have more to do with the phrasing and the fact that, as with any writer, large language models tend to repeat themselves.

That’s the logic behind AI-detection programs. The problem is that those systems are often AI-powered themselves, and they provide few details about how they arrived at their assessments. That makes them hard to trust.

A new feature from the AI-detection company Copyleaks, called AI Logic, provides more insight into not just whether and how much of something might have been written by AI, but what evidence it’s basing that decision on. What results is something that looks a lot like a plagiarism detector, with individual passages highlighted. You can then see whether Copyleaks flagged it because it matched text on a website known to be AI-generated, or if it was a phrase that the company’s research has determined is far more likely to appear in AI-produced than human-written text.

You don’t even necessarily have to seek out a gen AI tool to produce text with one these days. Tech companies like Microsoft and Google are adding AI helpers to workplace apps, but it’s even showing up in dating apps. A survey from the Kinsey Institute and Match, which owns Tinder and Hinge, found that 26% of singles were using AI in dating, whether it’s to punch up profiles or come up with better lines. AI writing is inescapable, and there are times when you probably want to know whether a person actually wrote what you’re reading. 

This additional information from a Copyleaks-checked text marks a step forward in the search for a way to separate the AI-made from the human-written, but the important element still isn’t the software. It takes a human being to look at this data and figure out what’s a coincidence and what’s concerning.

“The idea is really to get to a point where there is no question mark, to provide as much evidence as we can,” Copyleaks CEO Alon Yamin told me.

A noble sentiment, but I also wanted to see for myself what the AI detector would detect and why.

How AI detection works

Copyleaks started out by using AI models to identify specific writing styles as a way to detect copyright infringement. When OpenAI’s ChatGPT burst on the scene in 2022, the company realized it could use the same models to detect the style of large language models. Yamin called it “AI versus AI,” in that models were trained to look for specific factors like the length of sentences, punctuation usage and specific phrases. (Disclosure: Ziff Davis, CNET’s parent company, in April filed a lawsuit against OpenAI, alleging it infringed Ziff Davis copyrights in training and operating its AI systems.)

The problem with using AI to detect AI is that large language models are often a “black box” — they’ll produce an output that makes sense, and you know what went into training them, but they don’t show their work. Copyleaks’ AI Logic function tries to pull back the veil so people have a better sense of what in the copy they’re evaluating might actually be AI-written. 

“What’s really important is to have as much transparency around AI models [as possible], even internally,” Yamin said.

Read more: AI Essentials: 29 Ways to Make Gen AI Work for You, According to Our Experts

AI Logic uses two different approaches to identify text potentially written by an LLM. One, called AI Source Match, uses a database of AI-generated content from sources either created in-house by Copyleaks or on AI-produced sites online. This works much like a traditional plagiarism detector. “What we’ve discovered is that AI content, a lot of the time, if you ask the same question or a similar question over and over again, you’ll get similar answers or a similar version of the same answer,” Yamin said.

The other component, AI Phrases, detects terms and groups of words that Copyleaks’ research has determined are far more likely to be used by LLMs than by human writers. In one sample report, Copyleaks identified the phrase “with advancements in technology” as potentially AI-written. Copyleaks’ analysis of generated content found that the phrase appeared 125 times per million AI-written documents, compared with just six times per million documents written by people.

The question is, does it work?

Can Copyleaks spot AI content and explain why?

I ran a few documents through Copyleaks to see if AI Logic can identify what I know to be AI-created content, or if it flags human-written content as AI-written.

Example: A human-written classic

What better way to test an artificial intelligence tool than with a story about artificial intelligence? I asked Copyleaks to test a section of Isaac Asimov’s classic 1956 short story The Last Question, about a fictional artificial intelligence tasked with solving a difficult problem. Copyleaks successfully identified it as 100% matched text on the internet and 0% AI-written. 

Example: Partially AI-written

For this example, I asked ChatGPT to add two paragraphs of additional copy to a story I wrote and published earlier in the day. I ran the resulting text — my original story with the two AI-written paragraphs added at the bottom — through Copyleaks. 

Copyleaks successfully identified that 65.8% of this copy matched existing text (because it was literally an article already on the internet), but it didn’t pick up anything as being AI-generated. Those two paragraphs ChatGPT just wrote? Flew completely under the radar. 

Enlarge Image

Copyleaks thought everything in this article was written by AI, even though only a few paragraphs were.

Screenshot by Jon Reed/CNET

I tried again, this time asking Google’s Gemini to add some copy to my existing story. Copyleaks again identified that 67.2% of the text matched what was online, but it also reported that 100% of the text may have been AI-generated. Even text I wrote was flagged, with some phrases, like “generative AI model,” described as occurring more frequently in AI-written text. 

Example: Totally AI-written

In a test of generative AI’s ability to create things that are totally out of touch with reality, I asked it to write a news story as if the Cincinnati Bengals had won the Super Bowl. (In this fictional universe, Cincinnati beat the San Francisco 49ers by a score of 31-17.) When I ran the fake story through Copyleaks, it successfully identified it as entirely AI-written. 

Enlarge Image

Copyleaks’ AI Logic quickly realized this story about the Cincinnati Bengals winning the Super Bowl was written by an AI chatbot.

Screenshot by Jon Reed/CNET

What Copyleaks didn’t do, however, is explain why. It said no results were found in its AI Source Match or its AI Phrases, but with a note: “There is no specific phrase that indicates AI. However, other criteria suggest that this text was generated by AI.” 

I tried again, this time with a different ChatGPT-generated story about the Bengals winning the Super Bowl 27-24 over the 49ers, and Copyleaks provided a more detailed explanation. It calculated the content was 98.7% AI-created, with a handful of phrases singled out. These included some seemingly innocent terms like “made several critical” and “testament to years of.” It also included some strings of words that spread across multiple phrases or sentences, like “continues to evolve, the Bengals’ future,” which apparently occurred 317 times more frequently in the database’s AI-generated content than in human text documents. (After raising the issue with the first attempt with Copyleaks, I tried it again and got similar results to this second test.)

Just to be sure it wasn’t operating entirely on the fact that the Bengals have never won a Super Bowl, I asked ChatGPT to write an article about the Los Angeles Dodgers winning the World Series. Copyleaks found that 50.5% matched existing text online, but also reported it was 100% AI-written. 

A high-profile example

Copyleaks did some testing of its own, using a recent example of a controversial alleged use of AI. In May, the news outlet NOTUS said that a report from the Trump administration’s Make America Healthy Again Commission contained references to academic studies that did not exist. Researchers who were cited in the MAHA report told media outlets that they did not produce that work. Citations to nonexistent sources are a common result of AI hallucination, which is why it’s important to check anything an LLM cites. The Trump administration defended the report, with a spokesperson blaming “minor citation and formatting errors” and stating that the substance of the report remains unchanged. 

Copyleaks ran the report through its system, which reported finding 20.8% potential AI-written content. It found some sections around children’s mental health raised red flags in its AI Phrases database. Some phrases that occurred far more often in AI-written text included “impacts of social media on their” and “The Negative Impact of Social Media on Their Mental Health.”

Can an AI really detect AI-written text?

In my experience, the increased transparency from Copyleaks into how the tool works is a step forward for the world of AI detection, but this is still far from foolproof. There’s still a troubling risk of false positives. In my testing, sometimes words I had written just hours before (and I know AI didn’t play a role in them) could be flagged because of some of the phrasing. Still, Copyleaks was able to spot a bogus news article about a team that has never won a championship doing so. 

Yamin said the goal isn’t necessarily to be the ultimate source of truth but to provide people who need to assess whether and how AI has been used with tools to make better decisions. A human needs to be in the loop, but tools like Copyleaks can help with trust. 

“The idea in the end is to help humans in the process of evaluating content,” he said. “I think we’re in an age where content is everywhere, and it’s being produced more and more and faster than ever before. It’s getting harder to identify content that you can trust.”

Here’s my take: When using an AI detector, one way to have more confidence is to look specifically at what is being flagged as possibly AI-written. The occasional suspicious phrase may be, and likely is, innocent. After all, there are only so many different ways you can rearrange words — a compact phrase like “generative AI model” is pretty handy for us humans, same as for AI. But if it’s several whole paragraphs? That may be more troubling.

AI detectors, just like that rumor that the em dash is an AI tell, can have false positives. A tool that is still largely a black box will make mistakes, and that can be devastating for someone whose genuine writing was flagged through no fault of their own.

I asked Yamin how human writers can make sure their work isn’t caught in that trap. “Just do your thing,” he said. “Make sure you have your human touch.”