Improved support for IPv6
-------------------------
- make it possible to run SpamAssassin on an IPv6-only host:
affects installation, self-tests, rule updates, client, server,
and a command-line spamassassin
- Make sa-update and its infrastructure usable over IPv6 (Bug 6654)
- added options -4 and -6 to prefer/choose/force IPv4 or IPv6
in programs spamassassin, spamd, spamc, and sa-update
- if a module IO::Socket::IP is available, use it for network
communications regardless of a protocol family for DNS queries,
by spamd and in Mail/SpamAssassin/Client.pm. As a fallback when
IO::Socket::IP is unavailable, the IO::Socket::INET6 or eventually
the IO::Socket::INET are used
- allow spamd to simultaneously listen on multiple sockets, possibly
in different protocol domains (Unix sockets, INET or INET6 protocol
families)
- DnsResolver was updated allowing it to work on an IPv6-only host (Bug 6653)
- plugin RelayCountry now uses module Geo::IP and its database
of IPv6 addresses GEOIP_COUNTRY_EDITION_V6 if available
- command line options --listen and --allowed-ips in spamd now accept
IPv6 addresses
- the following configuration options were extended to accept IPv6
addresses: dns_server, trusted_networks, internal_networks, msa_networks,
(but not yet the whitelist_from_rcvd); their defaults were adjusted
accordingly
- parser of Received header fields can now deal with IPv6 addresses
- AutoWhitelist plugin can now deal with IPv6 addresses
- installation unit tests were updated to prevent them from failing
on an net6 -only host.
Optimizations
-------------
Several smaller performance optimizations were introduced,
among others: Bug 6508 (uses Net::Patricia if available),
Bug 6854 (base64 attachments), Bug 6915 (get_tag speedup).
New command-line options
-------------------------
- sa-update can now take multiple -v or --verbose options to increase
verbosity
- spamd: can listen on multiple sockets as documented at option --listen
- new sa-learn command option --max-size
New command-line options for spamd:
- added an option --listen (or -i), which can be specified multiple
times and allows spamd to accept requests over multiple INET (IPv4)
or INET6 (IPv6) or UNIX sockets; (Bug 6841); see also option --port
New command-line options for spamc:
- adding option -X (or --unavailable-tempfail) to allow spamc to return
EX_TEMPFAIL instead of EX_UNAVAILABLE when using option -x
As already noted in the IPv6 section, options -4 and -6 were added
to programs spamassassin, spamd, spamc, and sa-update.
New configuration options
-------------------------
Plugin/URIDNSBL: new tflags options 'a' and 'ns'
Plugin/AutoLearnThreshold: new option autolearn_force
Plugin/ASN: new options asn_prefix and clear_asn_lookups
Plugin/WLBLEval: new configuration options: enlist_uri_host, delist_uri_host,
with shorthands blacklist_uri_host and whitelist_uri_host and
an associated eval rule check_uri_host_listed
added configuration options dns_query_restriction (allow|deny) and
clear_dns_query_restriction (Bug 6884)
added sub-options dns0x20 and edns to the dns_options setting
added option dns_server which specifies an IP address of a DNS server
and optionally its port number.
added options dns_local_ports_permit, dns_local_ports_avoid and
dns_local_ports_none to control source port local ranges available to
DNS queries
added the following sub-options to the tflags setting:
autolearn_force, maxhits=N, ips_only, domains_only, a, ns.
the option whitelist_from_rcvd can now take an IP address as its second
argument (instead of a domain name), which can be useful for whitelisting
a sending mailer which has no reverse DNS mapping
ArchiveIterator: new options opt_max_size and opt_from_regex
added a tag (macro) _RULESVERSION_ which is a comma-separated list of
rules versions, retrieved from an '# UPDATE version' comment in rules files
New eval rules:
Plugin/BodyEval: check_body_length
Plugin/HeaderEval: check_equal_from_domains
Notable bug fixes
sa-update: avoid repeatedly downloading rules if subsequent unpacking
of rules and updating fails (Bug 6655)
fixed several incompatibilities with newer versions of Net::DNS
in sa-update and in the SpamAssassin library
Net::DNS bug [rt.cpan.org #83451]
Plugin/Razor2.pm: preserve entropy of a random numbers generator
across calls to Razor2 agent which clobbers it
added a workaround in BayesStore/MySQL.pm for MySQL server bug
( http://bugs.mysql.com/bug.php?id=46675 ):
fixed documentation: trailing dots in DNSBL zone names are not
required since version 3.1.0 of Mail::SpamAssassin (September 2005).
Compatibility - changes since 3.3.2:
A default setting for option 'dns_available' was changed from
'test' to 'yes' (Bug 6770, Bug 6769)
DNS queries generated by SpamAssassin now enable option EDNS0
in query packets and specify a buffer length of 4096 bytes.
This allows DNS replies larger than 512 bytes to be returned
in one packet, avoiding a need for re-issuing a failed query
using a TCP protocol. This default setting is well suited
if a DNS resolver (i.e. a recursive DNS server) is located
on the same LAN as a host running SpamAssassin (which is the
usual setup for all but perhaps a home use of SpamAssassin).
The but may not
be suitable if a DNS server is
- EDNS=4096
Bug 6862: Allow a DNS resolver to use EDNS
- bayes seen change
Dependency changes since 3.3.2:
- dropped dependency on a Perl module Net::Ident
- dropped dependency on a Perl module IP::Country::Fast (or IP::Country)
- new optional dependency on Geo::IP in a RelayCountry plugin (Bug 6599).
For backward compatibility IP::Country::Fast is used if Geo::IP is
not installed
- new optional dependency on IO::Socket::IP for a cleaner IP support
regardless of a protocol family (IPv4 or IPv6)
- new optional dependency on Net::Patricia to speed up lookups on
internal_networks, trusted_networks or msa_networks when these lists
contain a largen number of entries
- new optional dependency on a perl module Redis, which is used if
a Bayes data store backend is chosen
- new optional dependency on programs curl, wget, or a FreeBSD fetch;
sa-update will use any of these external programs to download rule
updates, either over IPv6 or over IPv4
- a perl module LWP::UserAgent as used by sa-update is now optional
if any of programs curl, wget, or fetch are available;
- minimal required version of NetAddr::IP was bumped to 4.010
Internal changes potentially affecting third party software
using Mail::SpamAssassin library
Bug 6690: Give a caller a choice to call srand() by itself or let a
SpamAssassin library do it. Controlled by option skip_prng_reseeding
in SpamAssassin.pm
Bug 6686: Allow Mail::SpamAssassin::parser to accept a message
also as a string ref, avoiding one copy in memory
Bug 6830: a caller may pass the original mail body size to
SpamAssassin through {suppl_attrib}{body_size}, which is
accessible to the eval rule check_body_length
Bug 6942: A new plugin callback "prefork_init" was introduced,
which should be called by a master process (e.g. spamd) before
forking multiple child processes. For compatibility this call is
currently optional, but recommended for new versions. Currently
only a Redis backend for Bayes checks will benefit from being
notified before a fork.
New plugins:
AskDNS.pm
multiple improvements and bug fixed in Plugin DCC.pm
+=item can(Name::Of::Package::function_name)
+This is a function call that returns C<1> if the perl package named
+C includes a function called C
+B that function returns a true value when called with no arguments,
+otherwise C is returned.
+Is similar to C, except that it also calls the named function,
+testing its return value (unlike the perl function UNIVERSAL::can).
+This makes it possible for a 'feature' function to determine its result
+value at run time.
=========================================
To: users, dev, announce
Subject: ANNOUNCE: Apache SpamAssassin 3.4.0 available
Release Notes -- Apache SpamAssassin -- Version 3.4.0
Introduction
------------
This is a major release. It introduces nearly two years of bug fixes and
features including the Bayes Redis (http://redis.io/) back-end (bug 6879),
eDNS changes (bug 6910), Native IPv6 Support, numerous URIBL.pm changes or
features and a small API change in libspamc (bug 6562) with many other
subtle changes.
IPv6 note: Besides testing on a dual-protocol hosts, this was also tested
on an IPv6-only host (works fine except Razor, using an external recursive
DNS server running on a dual-protocol host).
SpamAssassin was tested on perl 5.18.0, and (out of curiosity) also
on a Raspberry PI (ARM6, Raspbian / Debian 7.0 Wheezy, perl 5.14.2)
... yes it is 20 times slower than on an i7-960, but all tests pass!
Overall, this release has been tested in many production-level
environments for nearly a year. It is highly recommended and stable.
NOTE: Complete changes are available at
http://svn.apache.org/repos/asf/spamassassin/branches/3.4/Changes
Important Sendmail Bug
----------------------
Sendmail 8.14.5 and below contain a canonicalization bug that can cause
DKIM failures.
See https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6462.
Important Bayes Change
----------------------
Beyond the ability to use Redis as a Bayes backend server, Bug 5185
changed the way that message IDs were calculated for emails.
Because of this change, if you use Bayes and you are upgrading from a
version prior to 3.4.0, you should consider wiping your Bayes database
and starting fresh.
However, this is not mandatory. If you choose to keep your current
database tokens, these are the ramifications:
1 - If you re-process emails that have already been learned before, it
will create duplicate entries because of the new msg_id format.
The duplicates will expire, eventually, and should cause minimal
impact unless it occurs frequently.
2 - If you try and unlearn or reclassify an email processed prior to the
upgrade, the system will be unable to do so because of the new msg_id
format. If unlearning a message is important, consider just clearing
your Bayes store and starting from scratch.
Redis database backend for a Bayes database
-------------------------------------------
In addition to existing backends, the 3.4.0 introduces support for keeping
a Bayes database on a Redis server, either running locally, or accessed
over network. Similar to SQL backends, the database may be concurrently
used by several hosts running SpamAssassin.
The current implementation only supports a global Bayes database, i.e.
per-recipient sub-databases are not supported. Due to current limitations
in Redis 2.6.* server, accessing it is not possible over IPv6, but one may
chose connection over IPv4 or over a Unix socket. Bear in mind that Redis
server only offers limited access controls, so it is advisable to let the
Redis server bind to a loopback interface only, or to use other mechanisms
to limit access, such as local firewall rules.
The Redis backend can put a Lua scripting support in a Redis server to good
use, significantly reducing network packet rate and improving performance.
The Lua support is available in Redis server since version 2.6. In absence
of a Lua support, the Redis backend uses multiple traditional Redis command
so in principle it should work with a Redis server version 2.4, although
this is not recommended for busy sites.
Expiration of token and 'seen' message id entries is left to the Redis
server. There is no provision for manually expiring a database, so it is
highly recommended to leave the setting bayes_auto_expire to its default
value 1 (i.e. enabled).
Example configuration:
bayes_store_module Mail::SpamAssassin::BayesStore::Redis
bayes_sql_dsn server=localhost:6379
bayes_token_ttl 21d
bayes_seen_ttl 8d
bayes_auto_expire 1
Downloading and availability
----------------------------
Downloads are available from:
http://spamassassin.apache.org/downloads.cgi
md5sum of archive files:
568915b8d227eb5fc5d50abf8c10c35f Mail-SpamAssassin-3.4.0-rc2.tar.bz2
e653fd06387e15e24ab1d14ba81f473b Mail-SpamAssassin-3.4.0-rc2.tar.gz
d1880f5dc3d1a784d634b193bc83092a Mail-SpamAssassin-3.4.0-rc2.zip
de3929f93119523b5492fa9126cb2a68 Mail-SpamAssassin-rules-3.4.0-rc2.r1494509.tgz
sha1sum of archive files:
0863d52a5590f1fc22320c5631fb83ebb470e1fb Mail-SpamAssassin-3.4.0-rc2.tar.bz2
5aae5424de04978a0ca4f077be03a8c02b006397 Mail-SpamAssassin-3.4.0-rc2.tar.gz
88e31041894e5e05860253d055a13f5b1ef7e7aa Mail-SpamAssassin-3.4.0-rc2.zip
8ef4014b5e37d6f00c602d5c931879d17555e17a Mail-SpamAssassin-rules-3.4.0-rc2.r1494509.tgz
Note that the *-rules-*.tar.gz files are only necessary if you cannot, or do not
wish to, run "sa-update" after install to download the latest fresh rules.
See the INSTALL and UPGRADE files in the distribution for important installation notes.
GPG Verification Procedure
--------------------------
The release files also have a .asc accompanying them. The file serves
as an external GPG signature for the given release file. The signing
key is available via the wwwkeys.pgp.net key server, as well as
http://www.apache.org/dist/spamassassin/KEYS
The key information is:
pub 4096R/F7D39814 2009-12-02
Key fingerprint = D809 9BC7 9E17 D7E4 9BC2 1E31 FDE5 2F40 F7D3 9814
uid SpamAssassin Project Management Committee
uid SpamAssassin Signing Key (Code Signing Key, replacement for 1024D/265FA05B)
sub 4096R/7B3265A5 2009-12-02
To verify a release file, download the file with the accompanying .asc file and run the following commands:
gpg -v --keyserver wwwkeys.pgp.net --recv-key F7D39814
gpg --verify Mail-SpamAssassin-3.4.0-pre1.tar.bz2.asc
gpg --fingerprint F7D39814
Then verify that the key matches the signature.
Note that older versions of gnupg may not be able to complete the steps above.
Specifically, GnuPG v1.0.6, 1.0.7 & 1.2.6 failed while v1.4.11 worked flawlessly.
See http://www.apache.org/info/verification.html for more information on verifying Apache releases.
About Apache SpamAssassin
-------------------------
Apache SpamAssassin is a mature, widely-deployed open source project
that serves as a mail filter to identify spam. SpamAssassin uses a
variety of mechanisms including mail header and text analysis, Bayesian
filtering, DNS blocklists, and collaborative filtering databases. In
addition, Apache SpamAssassin has a modular architecture that allows
other technologies to be quickly incorporated as an addition or as a
replacement for existing methods.
Apache SpamAssassin typically runs on a server, classifies and labels
spam before it reaches your mailbox, while allowing other components of
a mail system to act on its results.
Most of the Apache SpamAssassin is written in Perl, with heavily
traversed code paths carefully optimized. Benefits are portability,
robustness and facilitated maintenance. It can run on a wide variety of
POSIX platforms.
The server and the Perl library feels at home on Unix and Linux
platforms, and reportedly also works on MS Windows systems under ActivePerl.
For more information, visit http://spamassassin.apache.org/
About The Apache Software Foundation
------------------------------------
Established in 1999, The Apache Software Foundation provides
organizational, legal, and financial support for more than 100
freely-available, collaboratively-developed Open Source projects. The
pragmatic Apache License enables individual and commercial users to
easily deploy Apache software; the Foundation's intellectual property
framework limits the legal exposure of its 2,500+ contributors.
For more information, visit http://www.apache.org/