Is that comment written by AI? What the research says. Seymour Digital Media blog header.

Is That Facebook Comment Written by AI? Here’s What the Research Says

You can’t prove a Facebook comment was written by AI. Nobody can. Not even the tools sold to do it. What you can do is spot patterns. When several show up at once, that’s a strong sign.

If you run a business page or a Facebook group, this matters. Fake comments, copy-and-paste praise and comments that lead to a sales pitch in your members’ inboxes all hurt the trust you’ve built. In this post we cover what the research says, which signs are worth watching, which ones fool people, and what to do about it. Every claim links to its source, so you can check it yourself.

Quick answer: You can’t prove it from one comment. Look for several signs together: a comment that could sit under any post, pet words in a cluster, chatbot leftovers like “I hope this helps,” and an account that comments the same way everywhere. Then ask a question only a real reader could answer.

First, the honest part: most people guess, and so do the tools

Before the checklist, here’s why no single sign should convince you.

Most people do no better than a coin flip. Researchers at Cornell and Stanford ran six tests with 4,600 people. They asked them to spot profiles written by AI for dating, Airbnb hosts and freelancers. People were right only 50 to 52% of the time. Paying them for right answers didn’t help. Telling them after each guess if they were right didn’t help either. (Jakesch, Hancock and Naaman, PNAS, 2023; free copy)

Short social posts are the hardest. A team at the National Research Council of Canada tested 250 people on short posts from X (once called Twitter). Some of the AI had been trained to write like a social media user. People caught that AI only 53.9% of the time. Detector tools struggled too. On those posts, GPTZero did no better than a coin flip. The best tool they tested got 61% right. (Dawkins, Fraser and Kiritchenko, 2025)

53.9% of short AI posts were spotted by people when the AI wrote like a social media user. Source: National Research Council of Canada, 2025.
Source: Dawkins, Fraser and Kiritchenko, National Research Council of Canada, 2025

Detector tools aren’t proof. OpenAI built its own tool to spot AI text. It shut the tool down in July 2023 because it wasn’t accurate enough. In OpenAI’s own tests, it caught 26% of AI text. It also wrongly flagged human writing 9% of the time. OpenAI said it wasn’t reliable on anything under 1,000 characters. That’s almost every Facebook comment. (OpenAI)

Detectors are hardest on people writing in a second language. A Stanford study ran seven popular detectors on 91 essays. Non-native English speakers wrote them for the TOEFL English test. On average, the detectors called 61.22% of the essays AI. All seven tools agreed on 18 of them. 89 of the 91 were flagged by at least one tool. Yet the same tools were close to perfect on essays by US Grade 8 students. (Liang et al., Patterns, 2023)

One group does well: people who use AI to write every day. In a 2025 study, five readers who often use AI for writing checked 300 articles. When they took a majority vote, they got only 1 of the 300 wrong. That beat most paid detector tools. It worked even when the AI text had been reworded to get past them. (Russell, Karpinska and Iyyer, 2025)

The takeaway: you get better with practice. But treat the signs below as clues to weigh, not proof.

Signs in the words

The best free guide we’ve found comes from Wikipedia’s volunteer editors. They’ve been cleaning up AI-written content since 2023. Their Signs of AI writing page is free to read. Here’s what applies to a Facebook comment.

Pet words, in groups. AI tools use some words far too often. The guide says one or two in a comment can be chance. Lots of them together is one of the strongest signs.

The pet words change with each new AI model. Wikipedia’s list for 2023 to mid-2024 includes “delve,” “crucial,” “pivotal,” “tapestry,” “testament” and “vibrant.” The guide says “delve” dropped off sharply in 2025. Its newest list, for mid-2026 on, includes “quietly,” “steady,” “practical,” “matters” and “leverage.” It also notes shorter sentences, about 8 to 20 words long. So if you’re still looking for “delve,” you’re looking for older AI.

AI pet words keep changing. 2023 to mid-2024: delve, crucial, pivotal, tapestry. Mid-2026 on: quietly, steady, practical, leverage. Look for a cluster of them, not one word on its own.
Source: Wikipedia, Signs of AI writing

Making ordinary things sound huge. AI likes to tell you something matters instead of showing why. Watch for “plays a pivotal role,” “stands as a testament” or “a key turning point,” mostly about something ordinary.

Empty “-ing” endings. Watch for sentences that end with “highlighting the importance of…” or “emphasizing the value of…” They sound smart but add nothing new.

Gushing like an ad. Words like “boasts,” “nestled,” “in the heart of” and “breathtaking” show up no matter what the topic is.

Unnamed experts. “Experts argue” or “observers have cited” with no names given.

Signs in the structure

“It’s not X, it’s Y.” Wikipedia’s guide flags lines like “Not only… but…” and “It is not just about…, it’s…” as common in AI writing. (Wikipedia)

Everything comes in threes. William Beutler of Beutler Ink, a Wikipedia strategist, sums up the guide. He points to AI’s habit of listing exactly three things, every time: three adjectives, three benefits or three takeaways. (Beutler Ink)

A neat little summary at the end. “Overall” or “In summary” to wrap up a comment that was only three sentences long.

Chatbot leftovers. Sometimes people paste the whole reply, including the parts the chatbot wrote to them. Wikipedia’s guide lists “I hope this helps,” “Of course!,” “Certainly!,” “Would you like…” and “let me know.” In a Facebook comment, that’s close to a confession.

Formatting that doesn’t fit a comment. Bold phrases, an emoji at the start of every line and Title Case Headings are all on Wikipedia’s list.

Em dashes: weak proof either way. Wikipedia notes that AI uses the long dash more than most people do. But in November 2025, OpenAI’s CEO Sam Altman said ChatGPT will now skip em dashes if users ask it to in their custom instructions. (Gizmodo) Lots of careful human writers use them too. A dash proves nothing. Neither does a missing one.

Signs in how people act (the most useful on Facebook)

Words only tell you so much in a two-line comment. The research points to two better places to look.

What helped, and what fooled people. In the Cornell and Stanford study, two clues really did point to AI: content that didn’t make sense, and content that repeated itself. Using just those two would have raised accuracy to 58.8%. Here’s what fooled people. They thought grammar mistakes meant AI, but mistakes were more likely to come from a human. They also took “I,” short forms like “don’t,” and mentions of family as proof of a human. None of those were reliable. (Jakesch, Hancock and Naaman, 2023)

Look at the account, not just the words. The National Research Council of Canada team said real-world checks should look at more than the text. Look at the account itself: its name, photo and how often it posts. And look at the network, such as groups of accounts that act together. (Dawkins, Fraser and Kiritchenko, 2025)

Here’s what that looks like on a Facebook page or group:

  • The comment could fit under any post. It praises or agrees but never mentions anything from yours.
  • It just repeats your post back to you. Your main point, reworded, with nothing added.
  • The same voice everywhere. Search the person’s name in your group, or scroll their recent activity. If every comment on every topic has the same shape and the same excitement, that’s a pattern.
  • Several accounts doing the same thing. New or empty accounts that join around the same time and comment in the same style.
  • No real reply. Ask a question only someone who read your post could answer, like “Which part, the second tip or the screenshot?” A real person answers the question. Silence or another vague reply tells you something.
Tip: ask a question only someone who read your post could answer. A real person answers it. Silence tells you something.

This isn’t a rare problem. LinkedIn says automated comment tools flood comment sections and crowd out real conversation. When it finds them, LinkedIn may leave those comments out of the “Most Relevant” view. It may also hide them from people outside the commenter’s network, and limit the account. (LinkedIn)

Not every AI comment is a bot

Some smooth, generic comments come from real people pressing a button Facebook gave them. In April 2024, TechRadar’s Lance Ulanoff went to wish a friend happy birthday. Facebook offered to “Comment with AI.” It let him pick a style (insightful, inspirational, heartfelt, casual or supportive) and edit the comment before posting. (TechRadar)

Others use AI because English isn’t their first language. The National Research Council of Canada team points out that AI can help people who aren’t fluent write clear messages. They also say detectors can be unfair to some groups. (Dawkins, Fraser and Kiritchenko, 2025) That matches the Stanford finding on second-language writers above.

61.22% of essays by non-native English writers were labelled AI by detectors, on average. Source: Liang et al., Stanford, 2023.
Source: Liang et al., Stanford, Patterns, 2023

So here’s the better question to ask: is a real person here, adding something, and engaging in good faith?

What to do about it on your page or group

  1. Look for several signs, never just one. One pet word, one dash or one polished sentence means nothing on its own.
  2. Don’t treat a detector score as proof. Even OpenAI pulled its own tool. And the research above shows detectors are least fair to people writing in a second language.
  3. Ask a specific follow-up question. It takes you ten seconds, and it’s the fastest test you have.
  4. Check what’s around the comment. Look at the account and its other activity before you decide.
  5. Screen people before they join. If you run a group, membership questions and Facebook’s Admin Assist can approve or decline join requests based on rules you set. That includes whether someone answered all the questions. Admins and moderators can also stop members from posting and commenting for a set time. (Meta)
  6. Put it in your group rules. A plain rule like “no automated or copy-and-paste comments” gives you something fair to point to when you remove one. LinkedIn already acts against comments posted with automation tools.
  7. Hold your own replies to the same standard. If you use AI to help write replies on your business page, edit them until they say something specific about the person you’re answering. Your customers are reading for the same signs.

Real engagement is the kind worth measuring

Comments, likes and shares only matter if real people are behind them. That’s true on Facebook, and it’s true in your reports. At Seymour Digital Media we help businesses sort the numbers that mean something from the noise. Then we build the Google Ads, SEO and tracking that bring in real customers.

Want to know what’s really working in your marketing? Book a 30-minute call and we’ll walk you through your data in plain English. Free, no commitment.

Sources

Similar Posts