Challenge–response spam filtering explained

A challenge–response (or C/R) system is a type of that automatically sends a reply with a challenge to the (alleged) sender of an incoming e-mail. It was originally designed in 1997 by Stan Weatherby, and was called Email Verification. In this reply, the purported sender is asked to perform some action to assure delivery of the original message, which would otherwise not be delivered. The action to perform typically takes relatively little effort to do once, but great effort to perform in large numbers. This effectively filters out spammers. Challenge–response systems only need to send challenges to unknown senders. Senders that have previously performed the challenging action, or who have previously been sent e-mail(s) to, would be automatically receive a challenge.

The challenge in challenge–response systems

C/R systems attempt to provide challenges that can be fulfilled easily for legitimate senders and non-easily for spammers. Two characteristics that differ between legitimate senders and spammers are exploited to achieve this goal:

Listed below are examples of challenges that are or could be used to exploit these differences:

Nowadays C/R systems are not used widely enough to make spammers bother to (automatically) respond to challenges. Therefore, C/R systems generally just rely on a simple challenge that would be made more complicated if spammers ever build such automated responders.

Recommendations for C/R systems

C/R systems should ideally:

Problems with sending challenges to forged email addresses can be reduced if the challenges are only sent when:

Criticisms

Critics of C/R systems have raised several issues regarding their legitimacy and usefulness as an email defense. A number of these issues relate to all programs which auto-respond to e-mail, including mailing list managers, vacation programs and bounce messages from mail servers.

Challenges sent to forged email addresses

Spammers can use a fake, non-existent address as sender address (in the field) in the e-mail header, but can also use a forged, existing sender address (a valid, but an arbitrary person's address without this person's consent). The latter would become increasingly common if e.g. callback verification would become more popular to detect spam. C/R systems challenging a message with a forged sender address would send their challenge as a new message to the person whose address was forged. This would create e-mail backscatter, which would effectively shift the burden from the person who would have received the spam to the person whose address was forged and which may be treated the same as any other Unsolicited Bulk Email (UBE) by the receiving system, possibly leading to blacklisting of the mail server or even listing on a DNSBL. In addition, if the forged sender decided to validate the challenge, the C/R user would receive the spam anyway and the forged sender address would be whitelisted.

Though definitely an undesirable side-effect, this issue would be non-existent if people, whose email address was used as a forged address in spam, happen to run a C/R system themselves. In this case, one of the C/R users must implement some form of return address signing (such as Bounce Address Tag Validation) to ensure that the challenge goes through. Also, if systems like SPF and DKIM became common, forged sender addresses would be recognized by these systems before reaching a C/R system.

In some cases, C/R systems can be tricked into becoming spam relays. To be useful, some part of the message under challenge is generally included in the challenge message. A spammer, knowing that they're sending to a C/R using system, could design their message so that their "spam payload" is in the part of the message that the challenge message includes. In this case, the forged sender is the actual recipient of the spam, and the C/R system unwittingly is the relay.

Social issues

Disseminating an ordinary email address that is protected by a C/R system results in challenges to those who send mail to that address. Some C/R critics consider it rude to give people your email address, then require them (unless previously whitelisted, which might not always be possible) to answer the challenge before they can send you mail.[3]

Advocates of C/R systems argue that the benefits by far outweigh the 'burden' of an incidental challenge, and that there will probably never be a final solution against spam without laying some kind of burden on the e-mail sender. They reason that the more widespread the use of C/R systems is, the more understood, accepted and appreciated they are. In an analogy with snail mail, the sender is prepared to pay for the stamp, in an analogy with phone calls, the caller is prepared to pay for the outgoing call.

Interaction with mailing lists or other automated mailers

Some C/R systems interact badly with mailing list software. If a person subscribed to a mailing list begins to use C/R software, posters to the mailing list may be confronted by challenge messages. Order confirmations, billing statements and delivery notices from online shopping systems are usually sent via automated systems. Email challenges sent to such systems can be lost, and legitimate mail sent by these systems may not reach the C/R system user.

Advocates of C/R systems argue that, though it takes extra effort, solutions for these problems exist if the end-user behind the C/R system does these simple things:

False positives

C/R advocates claim that such systems have a lower rate of false positives than other systems for automatically filtering unsolicited bulk email.

Critics argue that typical users of C/R systems still need to review their challenged mail regularly, looking for non-bulk mail or solicited bulk email for which the sender has not responded to the challenge. This issue is particularly notable with newsletters, transactional messages, and other solicited bulk email, as such senders do not usually check for challenges to their mail. However, if the bulk email in question was solicited, then the C/R user could be expected to have added it to the whitelist. If the bulk email was not solicited, then by definition it is spam, and is filtered by the C/R system.

Implementations

External links

Notes and References

  1. Recommendations for Automatic Responses to Electronic Mail

  2. Web site: Templeton. Brad. Proper principles for Challenge/Response anti-spam systems. 13 June 2014.
  3. Web site: Schryver. Vernon. Challenge/Response systems considered harmful. 13 June 2014.
  4. Web site: Legacy Communities - IBM Community.