I took a further look at charsets in mail, in respect to what I need to receive, and what spammers use. In theory, a mailer should always use the least necessary charset, us-ascii that is, unless the user types some non-ascii sign, in which case it should use the ISO-8859 charset if appropriate, or UTF8. If you configured it to use, say 8859-1, and you type an umlaut, it sets the charset to ISO-88859-1, if you type a cyrillic character, it should use UTF8. If you configured ISO-8859-5, it should use this for cyrillic, and UTF8 if you type an umlaut. Simple. So you only need us-ascii, ISO-8859 and UTF8. <\/p>\n
Now there are some braindead and\/or obsolete mail-programs, which use different and outdated charsets. As it happens, you will have contact to people whose mails appear in a hodge-podge of non-standardized charsets, most notably windows-125X. Now, for a western-european german-speaking context, where most mails are either german or english, with some very little french or spanish thrown in, I did some statistics regarding the charsets of spam and ham. <\/p>\n
This is a sample of 1220 legitimate Mails: <\/p>\n
\n83.6% iso-8859-1
\n5.3% us-ascii
\n4.6% utf-8
\n4.5% iso-8859-15
\n1.5% windows-1252\n<\/p><\/blockquote>\nThe rest half percent is negligible and consists of some other iso-8859 charsets. <\/p>\n
Now with spam, this looks quite different, the sample here are 6251 spam-mails:<\/p>\n
\n15.5% iso-8859-1
\n39.5% us-ascii
\n00.6% utf-8
\n00.1% iso-8859-15
\n14.7% windows-1252\n<\/p><\/blockquote>\nNow, where’s the rest? Its mostly a huge amount of the windows us-ascii replacement windows-1250, which is a completely superfluous charset:<\/p>\n
\n23.9% windows-1250
\n1.5% iso-2022-jp
\n1.3% koi8-r
\n1.1% iso-8859-2
\n0.5% windows-1255
\n0.3% windows-1254\n<\/p><\/blockquote>\nThe remaining percent are japanese, chinese and russian charsets, plus the remaining windows-125X-charsets. <\/p>\n
So now I can get rid of 25 percent of spam by not allowing chinese and other east-asian charsets, russian koi8 and most windows-125X charsets except windows-1252. I could get rid of another 14.7% by blocking this also, but that would piss of 1.5% of my legitimate contacts. <\/p>\n
The blocking can be accomplished by simply putting some rule into .procmailrc:
\n
\n:0:
\n* ^Content-Type.*windows-1250.*
\nspam
\n<\/code><\/p>\nOr by giving it a score in spamassassin
\n
\n\/\/begin
\nheader __ILLEGAL_CHARSET_1 Content-Type =~ windows-1250\/i
\nmeta ILLEGAL_CHARSETS (__ILLEGAL_CHARSET_1 + __ILLEGAL_CHARSET_2>= 1)
\nscore ILLEGAL_CHARSETS 3
\ndescribe ILLEGAL_CHARSETS foreign obsolete charsets
\n<\/code><\/p>\nOr by refusing it completely during transmission in postfix’ \/etc\/postfix\/headercheck.pcre
\n
\n\/^Content-Type:.*\\bcharset=\"?(?:
\n windows-1250 |
\n windows-1251 |
\n windows-1253 |
\n windows-1254 |
\n windows-1255 |
\n windows-1256 |
\n windows-1257 |
\n windows-1258 |
\n windows-874)\\b\/ REJECT Illegal Charset
\n<\/code><\/p>\n\/etc\/postfix\/headercheck.pcre can also be used to block very selectively; if for instance an “unsubscribe” does not get honoured (it’s of course useless against normal spammers since they constantly change addresses):
\n
\n\/^From: .*test@example.com\/ REJECT I said NO, you spamming moron
\n<\/code><\/p>\nAnd if you want to test a rule, save an email as “testcase” or something and check if it matches:
\n
\npostmap -q - pcre:\/etc\/postfix\/headercheck.pcre < testcase\n<\/code>
\n<\/code><\/p>\n","protected":false},"excerpt":{"rendered":"I took a further look at charsets in mail, in respect to what I need to receive, and what spammers use. In theory, a mailer should always use the least necessary charset, us-ascii that is, unless the user types some non-ascii sign, in which case it should use the ISO-8859 charset if appropriate, or UTF8. […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[],"class_list":["post-20","post","type-post","status-publish","format-standard","hentry","category-computers"],"_links":{"self":[{"href":"https:\/\/seegras.discordia.ch\/Blog\/wp-json\/wp\/v2\/posts\/20","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/seegras.discordia.ch\/Blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/seegras.discordia.ch\/Blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/seegras.discordia.ch\/Blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/seegras.discordia.ch\/Blog\/wp-json\/wp\/v2\/comments?post=20"}],"version-history":[{"count":3,"href":"https:\/\/seegras.discordia.ch\/Blog\/wp-json\/wp\/v2\/posts\/20\/revisions"}],"predecessor-version":[{"id":122,"href":"https:\/\/seegras.discordia.ch\/Blog\/wp-json\/wp\/v2\/posts\/20\/revisions\/122"}],"wp:attachment":[{"href":"https:\/\/seegras.discordia.ch\/Blog\/wp-json\/wp\/v2\/media?parent=20"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/seegras.discordia.ch\/Blog\/wp-json\/wp\/v2\/categories?post=20"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/seegras.discordia.ch\/Blog\/wp-json\/wp\/v2\/tags?post=20"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}