This is a repost of my own essay, Challenge-Response Anti-Spam Systems Considered Harmful. Following a really pathetic thread on debian-user, I finally wrote an essay I'd been kicking around for a year or so.
Spam is a growing, heck, exploding problem. No doubt. Challenge-response
(C-R) is a flawed tactic, for the following reasons.
0. Weak, and trivially abused, verification basis.
Even where used, C-R systems are readily bypassed by spammers.
The 'FROM:' header of email can be, and routinely is, spoofed. It offers
no degree of authentication or evidence of identity.
C-R uses the "From:" header (with implementation-specific variations) as
an authentication key. While a given key is going to have a relatively
low likelihood of being cleared by a given user, there are keys which
will have a high likelihood of being cleared. Off the top of my head,
@microsoft.com, @aol.com, @ebay.com, @*.gov, and other major commercial,
financial, and governmental institutions, would be likely to be cleared
by a large number of users. Similar "social engineering" tactics are
already used by spammers.
C-R moves you back to square one of the fact that SMTP can't provide
authentication of email headers. At the very least, contextual analysis
of headers (as Alan admits) is necessary. If you're already taking this
step, heuristic and Bayesian methods are a low-overhead next step, which
have proven to be highly effective and accurate.
By contrast, systems which utilize multiple metrics -- sender,
header integrity, content, context, Bayesian analysis -- provide
a broader, deeper, richer set of metrics on which to gauge spam.
1. Mistaken interpretation of anti-spam goals
The intent of a practical anti-spam system is not to ensure, at all
costs, that no spam should darken the reader's inbox at any cost. If
that's the goal, then unplugging your computer is the simplest fix.
At a practical level, the goal is to minimize the amount of spam
received, while ensuring no (or the very minimum) of legitimate mail is
lost. Inconvieniencing spammers is a plus. It is currently possible to
achieve rates of a very small handful of spam messages per week via a mix
of whitelisting and content-filtering systems, with Bayesian filters
attaining very high and accurate rates.
C-R systems in practice achieve an unacceptably high false-positive rate
(non-spam treated as spam), and may in fact be highly suseptible to
false-negatives (spam treated as non-spam) via spoofing.
2. Misplaced burden.
Effective spam management tools should place the burden either on the
spammer, or at the very least, on the person receiving the benefits of
the filtering (the mail recipient). Instead, challenge-response puts the
burden on, at best, a person not directly benefitting, and quite likely
(read on) a completely innocent party. The one party who should be
inconvenienced by spam consequences -- the spammer -- isn't affected at
3. Privacy violation.
A record of our correspondence is being maintained by a third party who
has no business knowing of the transaction. Many people will refuse to
respond to C-R requests for this reason.
Virtually all C-R systems must be implemented on the mail server --
putting them effectively _out_ of the immediate reach of the casual home
email user, and putting critical information of the email habits of both
yourself and your correspondents in the hands of a third party.
Most of the _general_ discussion (that is, outside this mailing list) has
concerned service-model enterprise models in which C-R is provided and
hosted by a third-party, which is then aquiring a rather interesting
database of communications patterns, which _must_ be maintained on a
persistent basis. Not the sort of thing I'd like to have available to an
arbitrary subpeona request.
4. Less effective at greater burden than reciever-side whitelisting.
A C-R system is essentially an outsourced whitelist system. The
difference between a C-R system and a self-maintained whitelist is that
the latter is:
Maintained by the mail recipient, rather than a third party service
Is the responsibility of the mail recipient, rather than the sender.
Places the burden on the recipient to add new addresses to allow/deny
I might add that I myself use a mix of whitelisting and spam filtering
(via SpamAssassin) to filter my own mail with a very high level of
accuracy, in terms of true positives, true negatives, false positives,
and false negatives. Namely: better than 98% true positive (filtered
spam), less than 2% false negative (unfiltered spam), 99.98% true
negative (unfiltered non-spam), and less than 0.02% false positive
(filtered non-spam). While some C-R proponents claim filtering doesn't
work, it clearly does.
5. High type II error (beta).
Because of numerous issues in sender-compliance with C-R systems, C-R
tends to a high false postive rate. This is known as type II error, in
statistical tests, and is denoted by beta.
The mechanics of C-R systems lead to a fairly high probability that users
of such systems will find themselves missing an unacceptably high rate of
non-spam (aka "ham") mail, possibly with very high significance (e.g.:
client, commercial prospect, or family communications).
In a staggering display of transrational behavior, C-R proponents
frequently and vociferously blame this failure of C-R on the
unwillingness of bystanders to be drawn into the misguided system.
C-R systems assume all mail to be spam until proven otherwise. A rational
system assumes mail to be of _unknown_ quality, until determined to be
spam or non-spam. If mail processing can't determine the mail's quality,
it is treated as "grey". Such "greymail" generally amounts to a small
handfull of messages daily, even for heavy mail users, and can be readily
evaluated, with whitelists and spam filters trivially updated.
For a description of statistical type II errors, see:
6. Potential denial of service.
C-R systems can be used intentionally or otherwise in a denial-of-service
or "Joe Job" attack on an innocent third party. In fact, this is likely
to start happening shortly as C-R becomes more widespread.
How? Simply: Spammer spoofs a legitimate sending address (this is already
commonplace). C-R systems then send out a challenge to this address. With
only 1% penetration of C-R, the victim of the C-R/Spam attack is deluged
with 100,000 challenge emails. This could likely lead to lawsuits or
other legal challenges.
7. C-R - C-R deadlock
This is almost funny.
How do two C-R system users ever start talking to each other?
User A sends mail to user B. While user B's address is then known to A,
user B's C-R server's mail is not.
User B's C-R system sends a challenge to A...
...who intercepts the challenge with A's C-R system, which sends a
challenge to user B's C-R system...
Rinse, wash, repeat....
No, I didn't think this one up myself, see Ed Felton's "A Challenging
Response to Challenge-Response"
Bypassing this deadlock then opens an obvious loophole for spammers to
While _some_ C-R systems may avoid this particular pitfall, current
experience with vacation responders and spam-notification filters provide
strong empirical evidence that a significant number of C-R systems will
in fact _not_ get this right.
This and several following issues are often countered with "but a
well-designed C-R system won't do that". Unfortunately, there will be,
and are, many poorly-designed C-R systems.
8. Potential integration into spam email harvest systems.
One commonplace piece of advice for avoiding spam is to not respond to
opt-out, aka email validation testing, requests.
C-R spoofing on the part of spammers would simply hijack a presumption
that C-R requests were valid to provide spammers with higher-quality
mailing lists. See the current rash of identity theft / CC theft scams
based on "updating your account information".
C-R at best promotes bad personal identity protection practices.
9. Likely consequences: C-R messages and users blacklisted or
The C-R user is likely to find their own address added to blocklists from
many users and/or mailing list adminstrators burned by malformed, or
simply unwanted, C-R requests. Simply: people who receive such
requests are very likely to just add the sending address, or user
corresponding to the request, to their own personal blacklists.
This is my own current M.O. with C-R requests, and andecdotal
evidence suggests it's a common practice.
This factor is entirely outside the bounds of the C-R system; it is a
reflection of the independent response of individuals and organizations
to receiving C-R challenges. C-R definitionally cannot accomodate this.
Another possibility is that, due to user concensus, spam filters
simply tag C-R messages as spam, either with a direct rule or as a
result of Bayesian weighted scoring.
Beyond any semiotic arguments of what spam is or isn't, if the
operational reality is that Spamassassin reflects the opinion of SA users
and developers and treats C-R transactions as spam, it is for all
10. Mailing list burden.
C-R systems typically misfunction on mailing lists in one of two ways,
neither of which is acceptable:
The C-R sends a challenge to the list for messages received.
The C-R sends a challenge to each individual listmember for the first
In both cases, the burden is placed on a party who could care less about
the benefits of the C-R system. Several lists of my aquaintance have
taken to permanently banning any users who exhibit use of misconfigured
11. Fails to address techno-economic underpinnings of spam.
Spam exists for one reason: it's profitable.
It's profitable because technology allows the costs of sending a large
number of mail messages to be lower than the revenues available for doing
Any effective spam remedy must attack one or the other side (or both) of
this equation: raise the costs or reduce the technological effectiveness,
on the one side, or reduce revenues on the other.
C-R, as with most recipient-side filtering systems, imposes negligible
incremental overhead on the spammer. A delivery is made, the spam server
moves on, the cost is a single SMTP connection for a fractional second.
Collateral costs are high: for legitimate senders, spoofed reply
addresses, mailing lists, and retaliatory actions on the C-R user.
A truly effective spam defense must attack the techical and economic
aspects, in as unobtrusive a manner as possible.
The one system which seems to best fit this requirement is the Teergrub
-- the spam tar-baby, FAQ at:
A teergrubing mailserver costs a spammer multiple SMTP connections, an
inherently finite resource, for possibly hours. Workarounds on the part
of the spammer are possible, but all result in higher costs, reduced
delivery, or both. The net effect is essentially a delivery payment
requirement, though the payment is in the form of time and configuration
on the part of the spammer. Collateral damage is low -- if a teergrube
_does_ unintentionally filter a legitimate sender, the only cost is a
single (or very small number of) delayed delivery. This and other issues
are covered at the FAQ above, read it before posing hypothetical
Karsten M. Self
SCO -- backgrounder on Caldera/SCO vs IBM
Support the EFF!!
There is no K5 cabal.