LayerSift
Log inStart free

← All docs

Protecting forms from spam

Comment threads, reviews, contact forms, listings, sign-up bios: anywhere a visitor can write something, a script will post what it is paid to post (OWASP calls it spamming, OAT-017). It comes as a campaign — one message, or a spinner's variants of it, from address after address — so each address posts once or twice and stays under any per-address limit. What gives it away is that the message is the same.

An assessment with content — what was written — sees that sameness, and remembers what your moderators decided.

What is being watched

  • One message from many senders. Every message is fingerprinted and compared with the last hour's. Three addresses posting it, or one near enough, is a reason (content:repeated, 30 — 45 with a link in it, which is what spam is for); ten is a challenge on its own (60). Near enough means sharing at least half their phrases: a link retagged with a new tracking id is the same message, and so is a variant with a greeting or a few words swapped.
  • A message your moderators reported. Report a message as spam and its close copies are known for seven days, from wherever they are posted next (content:reportedSpam, 60: a challenge on its own, a block beside anything else). Close means sharing four fifths of their phrases or more — closer than a campaign is counted by — and linking to the same hosts: a petition with a spammer's link added is near the petition, and reporting it must not report the petition. Where verdicts disagree, the nearest message's stands.
  • A message your moderators approved. A petition or a form letter that many real people copy looks like a campaign. Approve one and its near-duplicates are not held against their senders for thirty days (approved).
  • Many posts from one address. Thirty in an hour is a reason (velocity:postsFromAddress, 30), a hundred a challenge (60). From one device (velocity:postsFromDevice), at the same marks: one browser behind a proxy pool is an address per post, and one device for all of them, counted by the session token's visitor. Seen both ways, it is counted once. Across an IPv6 /48 (velocity:postsFromNetwork), at sixteen times those marks: a fresh /64 of a tunnel broker's free /48 for every few posts, unless the devices that ran the agent there make it a campus's or a carrier's.
  • Many links. Five or more in one message is a reason (content:manyLinks, 15).
  • The client, as for every assessment.

A message under eight words is too short to tell a campaign from a common phrase — "thanks, great post" — and is never counted as repeated. A message is read as a reader sees it: characters that are not drawn at all — zero-width spaces, soft hyphens — are dropped, and Cyrillic and Greek letters drawn like Latin ones are read as those, so a spinner sprinkling invisible characters through a message, or writing "сheap" with a Cyrillic с, has not written a new one.

The sameness is the message's, so it falls on everyone who posts it. A receipt from identifying does not excuse it (a spam farm's browsers earn those), but a receipt earned by passing a challenge does: whoever passed is not asked again for posting a popular message. A message moderators reported is excused by nothing.

What is kept

Never the text. Each message is read in memory and kept for two hours as a MinHash: its runs of three words hashed 24 ways under a key of this deployment, keeping the smallest of each — 24 numbers that estimate how much two messages have in common, and give back nothing of either. Without the key, not even whether a given text was posted can be asked of them. A moderator's verdict is kept the same way: seven days for spam, thirty for an approval.

How near is near is a trade, measured. On a spinner's variants of a forty-word message the share of phrases in common ran from 0.6 to 1.0, and between unrelated messages — including two that open with the same greeting — a tenth or less; the threshold is half. Over 400 keys, a variant with a link retagged or a greeting swapped was found every time, one with two phrases reworded 99 times in 100, and one with its whole opening rewritten 94. A short message (fifteen or twenty words) with a word changed is found most of the time rather than always. Two different messages from one template ("Is the shop open on Sunday? I would like to…") share about four tenths and are rarely counted together — which is still why three senders is only a reason, and ten a challenge a person can pass.

At the edge, no code

With the edge enforcer in front of the site, name the forms and the field that carries each message:

"ANTIBOT_CONTENT_FIELDS": "/reviews=review.body,/contact=message"

Every POST to them is assessed with what was written — read from a copy of the request, nothing else of the body — and the verdict reaches your application in antibot-assess-* headers, so a challenge can hold a message for moderation rather than refuse it. Moderators' verdicts still come from your server, with reportContentOutcome.

From your server

verdict = antibot.assess(
    ip=client_ip,
    headers=raw_header_pairs,
    method="POST",
    url="https://shop.example/reviews",
    content=form["review"],
    session_token=request.cookies.get("_abid"),
)
if verdict["action"] == "block":
    abort(403)
if verdict["action"] == "challenge":
    hold_for_moderation()      # or show a challenge before accepting it

And when a moderator decides, with the address the message was posted from:

antibot.report_content_outcome(ip=review.ip, content=review.text, outcome="failure")   # spam
antibot.report_content_outcome(ip=review.ip, content=review.text, outcome="success")   # not spam

The same in every server SDK: assess with content, and reportContentOutcome. The response carries:

"content": {
  "words": 41, "links": 1, "linkHosts": 1,
  "sendersOfContent": 6, "postsFromAddress": 1, "postsFromDevice": 0, "postsFromNetwork": 0,
  "reportedSpam": false, "approved": false
}

and rules read every field as content.*. The editor has Hold back repeated messages with links (three senders and a link, unless approved: a challenge, once — whoever passed it is not asked again, as the scoring does not ask them) and Block messages moderators reported as spam.

Knowing it is happening

Edge traffic names the reasons, and the protection checklist on the Proxy page says whether messages are being assessed and moderators' verdicts reported — the verdicts are what turn one moderator's click into every copy caught.