Running your own mail server gives you complete control over your email — no data mining, no storage limits, no dependency on third-party services. But with that control comes responsibility: you need to filter spam, prevent your server from being used to send spam, and ensure your emails are actually delivered to recipients.

This guide covers setting up Postfix as a Mail Transfer Agent (MTA) and SpamAssassin as a spam filter on Ubuntu Server 24.04.

Why server-side spam filtering matters

Spam filtering at the server level catches unwanted email before it reaches any mailbox. This is more efficient than client-side filtering (filtering in your email client) for several reasons:

  • Server filtering runs once, protecting all users and all devices simultaneously
  • Server-level rejection refuses spam at the connection stage, before even accepting the email
  • It reduces storage usage by not storing spam
  • Consistent filtering rules apply everywhere, even on mobile clients

SpamAssassin uses multiple techniques simultaneously: rule-based scoring (checking patterns in email headers and body), DNS-based blacklists (checking if the sending IP is known for spam), Bayesian analysis (learning from previously marked spam), and collaborative databases. The combination makes it very effective at distinguishing legitimate email from spam.

What you need before starting

Before setting up Postfix and SpamAssassin, verify you have these prerequisites:

A VPS with a clean IP address: Home ISPs typically block port 25 (the SMTP port used for mail delivery) and may use IP ranges with poor reputation. A VPS from Hetzner, DigitalOcean, Vultr, or a similar provider with a clean IP is strongly recommended. Expect to spend $5-15/month.

A domain name: You need a domain you own and control (e.g., example.com). You will add DNS records to this domain.

Reverse DNS (PTR record): Most VPS providers let you set the reverse DNS for your IP through their control panel. Set it to your mail server hostname (e.g., mail.example.com). This is critical for deliverability — many receiving servers reject email from IPs without correct reverse DNS.

Open ports: Ensure your server’s firewall allows ports 25 (SMTP, for receiving mail), 587 (submission, for sending mail from clients), and 993 (IMAPS, if you plan to add Dovecot for IMAP access).

For general Linux news, tutorials and practical guides covering server administration and infrastructure topics, i-actu.fr regularly publishes accessible content for system administrators working with Linux and open-source tools.

Installing Postfix

Postfix is a Mail Transfer Agent — the software that receives, routes, and delivers email between servers.

Install Postfix:

sudo apt update
sudo apt install postfix

During installation, a configuration dialog appears. Choose:

  • General type of configuration: Internet Site
  • System mail name: your fully qualified domain name (e.g., mail.example.com)

After installation, Postfix is running with basic defaults.

Verify Postfix is running:

sudo systemctl status postfix
sudo postfix status

View the current configuration:

postconf -n    # shows non-default configuration values

Configuring Postfix as an SMTP server

The main Postfix configuration file is /etc/postfix/main.cf. Edit it to set up your mail server properly:

sudo nano /etc/postfix/main.cf

Key settings to configure:

# Your server's hostname
myhostname = mail.example.com

# Your domain name
mydomain = example.com

# Use your domain in outgoing mail addresses
myorigin = $mydomain

# Accept mail for these domains
mydestination = $myhostname, $mydomain, localhost.$mydomain, localhost

# Networks allowed to relay mail
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128

# Mail delivery location
home_mailbox = Maildir/

# Enable SASL authentication for submission
smtpd_sasl_auth_enable = yes
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth

# TLS settings (replace with your certificate paths)
smtpd_tls_cert_file = /etc/letsencrypt/live/mail.example.com/fullchain.pem
smtpd_tls_key_file = /etc/letsencrypt/live/mail.example.com/privkey.pem
smtpd_use_tls = yes
smtpd_tls_auth_only = yes

# TLS security level for outbound mail
smtp_tls_security_level = may
smtp_tls_loglevel = 1

Apply the configuration:

sudo postfix reload

Enable the submission port (587) for sending mail from email clients:

Edit /etc/postfix/master.cf and uncomment the submission lines:

sudo nano /etc/postfix/master.cf

Find and uncomment (remove the #):

submission inet n       -       y       -       -       smtpd
  -o syslog_name=postfix/submission
  -o smtpd_tls_security_level=encrypt
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_tls_auth_only=yes
  -o smtpd_reject_unlisted_recipient=no
  -o smtpd_recipient_restrictions=
  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject

Reload Postfix:

sudo postfix reload

Installing SpamAssassin

SpamAssassin is the spam scoring engine. Install it along with its daemon (spamd) for efficient processing:

sudo apt install spamassassin spamc

Enable and start the daemon:

sudo systemctl enable spamassassin
sudo systemctl start spamassassin

Create a dedicated SpamAssassin user (running as root is a security risk):

sudo adduser --no-create-home --disabled-login --gecos "" spamd

Edit the SpamAssassin configuration:

sudo nano /etc/spamassassin/local.cf

Add these basic settings:

# Set required score for spam classification
required_score 5.0

# Rewrite email subjects of spam
rewrite_header Subject *SPAM*

# Add spam score headers to all messages
add_header all Status _YESNO_, score=_SCORE_ required=_REQD_ tests=_TESTS_
add_header spam Flag _YESNO_

# Enable Bayes database (learns from marked spam/ham)
use_bayes 1
bayes_auto_learn 1

# Use DNS-based blacklists
use_dcc 0
use_razor2 0

# Check sender's reputation
skip_rbl_checks 0

Update SpamAssassin rules:

sudo sa-update
sudo systemctl restart spamassassin

Schedule daily rule updates via cron:

sudo crontab -e

Add:

0 2 * * * /usr/bin/sa-update --no-gpg && /usr/bin/systemctl restart spamassassin

Integrating SpamAssassin with Postfix

The integration uses a content filter — Postfix passes incoming mail through SpamAssassin before delivery.

Method 1: Using the spamc client in Postfix

Edit /etc/postfix/master.cf:

sudo nano /etc/postfix/master.cf

Add after the smtp line:

smtp      inet  n       -       y       -       -       smtpd
  -o content_filter=spamassassin

spamassassin unix -     n       n       -       -       pipe
  user=spamd argv=/usr/bin/spamc -f -e /usr/sbin/sendmail -oi -f ${sender} ${recipient}

Reload Postfix:

sudo postfix reload

Method 2: Using amavisd-new (more powerful, handles viruses too):

sudo apt install amavisd-new spamassassin
sudo systemctl enable amavis

Edit /etc/amavis/conf.d/15-content_filter_mode:

sudo nano /etc/amavis/conf.d/15-content_filter_mode

Uncomment to enable spam checking:

@bypass_spam_checks_maps = (
  \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re);

Change to enable (remove the @bypass_spam_checks_maps line and uncomment):

# Enable spam checking

Configure Postfix to use amavis in main.cf:

content_filter = smtp-amavis:[127.0.0.1]:10024

And in master.cf:

smtp-amavis unix -      -       n       -       2       smtp
  -o smtp_data_done_timeout=1200
  -o smtp_send_xforward_command=yes
  -o disable_dns_lookups=yes
  -o max_use=20

127.0.0.1:10025 inet n  -       n       -       -       smtpd
  -o content_filter=
  -o local_recipient_maps=
  -o relay_recipient_maps=
  -o smtpd_restriction_classes=
  -o smtpd_delay_reject=no
  -o smtpd_client_restrictions=permit_mynetworks,reject
  -o smtpd_helo_restrictions=
  -o smtpd_sender_restrictions=
  -o smtpd_recipient_restrictions=permit_mynetworks,reject
  -o mynetworks=127.0.0.0/8
  -o strict_rfc821_envelopes=yes

Configuring spam thresholds

SpamAssassin assigns each email a numeric spam score. You control what happens based on this score in your SpamAssassin configuration (/etc/spamassassin/local.cf):

# Score at which an email is considered spam
required_score 5.0

# Reject email with a very high score (these are almost certainly spam)
# Use in Postfix: add milter or check_policy_service
# Or handle in delivery agent:
report_safe 1      # 0: add headers only, 1: convert to attachment, 2: no original

# Auto-learn thresholds
bayes_auto_learn_threshold_nonspam 0.1
bayes_auto_learn_threshold_spam 12.0

A score of 5.0 is the standard default — messages scoring below 5.0 are delivered normally, above 5.0 are marked as spam. You can:

  • Deliver marked spam to a Spam folder (configured in your delivery agent, like Dovecot/Procmail)
  • Reject high-scoring spam at the SMTP level
  • Silently discard spam above a very high threshold (not recommended — you will lose legitimate email)

For a balanced approach, deliver spam-tagged email to a spam folder with a score of 5.0+, and reject at the SMTP level for scores above 15.0.

Testing your spam filter

Test SpamAssassin directly with the GTUBE test message:

The GTUBE (Generic Test for Unsolicited Bulk Email) is a standardized test string that SpamAssassin always flags as spam:

echo "Subject: Test

XJS*C4JDBQADN1.NSBN3*2IDNEN*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL*C.34X" | spamc

The output shows the spam score. GTUBE should score very high (1000+).

Test a legitimate email:

echo "Subject: Hello

This is a normal email to test spam filtering." | spamc

This should score very low (under 1.0).

Send a test email through Postfix:

echo "Test message" | sendmail -v your@email.com

Check /var/log/mail.log to see the email flow:

sudo tail -f /var/log/mail.log

Check SpamAssassin headers on received email:

After receiving a message through your server, view the raw email headers. You should see lines like:

X-Spam-Status: No, score=0.8 required=5.0 tests=...
X-Spam-Score: 0.8

Setting up SPF, DKIM and DMARC

These three DNS records are essential for email deliverability in 2026. Without them, your outgoing email will be rejected or marked as spam by Gmail, Outlook and other major providers.

SPF (Sender Policy Framework) — tells receiving servers which IP addresses are allowed to send email for your domain.

Add a TXT record to your DNS for example.com:

v=spf1 ip4:YOUR.SERVER.IP.ADDRESS -all

The -all at the end means “reject mail from any server not listed”. Use ~all (soft fail) initially while testing.

DKIM (DomainKeys Identified Mail) — adds a cryptographic signature to outgoing emails.

Install OpenDKIM:

sudo apt install opendkim opendkim-tools

Generate a key pair:

sudo mkdir /etc/opendkim/keys
sudo opendkim-genkey -t -s mail -d example.com -D /etc/opendkim/keys/example.com/
sudo chown -R opendkim:opendkim /etc/opendkim/keys/

The public key to add to DNS is in /etc/opendkim/keys/example.com/mail.txt. Add it as a TXT record at mail._domainkey.example.com.

Configure OpenDKIM:

sudo nano /etc/opendkim.conf
AutoRestart             Yes
AutoRestartRate         10/1h
Umask                   002
Syslog                  yes
LogWhy                  Yes
Canonicalization        relaxed/simple
ExternalIgnoreList      refile:/etc/opendkim/TrustedHosts
InternalHosts           refile:/etc/opendkim/TrustedHosts
KeyTable                refile:/etc/opendkim/KeyTable
SigningTable            refile:/etc/opendkim/SigningTable
Mode                    sv
PidFile                 /run/opendkim/opendkim.pid
SignatureAlgorithm      rsa-sha256
Socket                  local:/run/opendkim/opendkim.sock

Connect OpenDKIM to Postfix by adding to /etc/postfix/main.cf:

milter_protocol = 6
milter_default_action = accept
smtpd_milters = local:opendkim/opendkim.sock
non_smtpd_milters = local:opendkim/opendkim.sock

DMARC (Domain-based Message Authentication, Reporting and Conformance) — tells receiving servers what to do if SPF or DKIM checks fail.

Add a TXT record at _dmarc.example.com:

v=DMARC1; p=quarantine; pct=100; rua=mailto:dmarc@example.com; ruf=mailto:dmarc@example.com

Start with p=none (monitoring only) to see reports without affecting delivery, then move to p=quarantine (mark failing email as spam), and finally p=reject (reject failing email) once you are confident your configuration is correct.

Monitoring and tuning

After your mail server is running, monitor it regularly.

Check mail logs for delivery status:

sudo tail -f /var/log/mail.log
sudo grep "reject" /var/log/mail.log | tail -20    # see what was rejected
sudo grep "spam" /var/log/mail.log | tail -20       # spam filtering activity

Monitor your IP reputation:

Check your IP against major blacklists periodically using mxtoolbox.com or multirbl.valli.org. A blacklisted IP causes most of your outgoing email to be rejected.

Train SpamAssassin’s Bayes filter:

When you receive spam that SpamAssassin missed, teach it:

sa-learn --spam /path/to/spam-message.eml

When SpamAssassin incorrectly flags a legitimate email:

sa-learn --ham /path/to/legitimate-message.eml

After training, rebuild the Bayes database:

sa-learn --rebuild

Check SpamAssassin statistics:

sa-learn --dump magic

This shows how many messages are in the Bayes database. The filter becomes more accurate as the database grows — aim for at least 200 spam and 200 ham messages.

Review SpamAssassin hit rates by analyzing mail logs:

sudo grep "spam" /var/log/mail.log | grep -v "not spam" | wc -l    # spam count
sudo grep "not spam" /var/log/mail.log | wc -l                       # ham count

Running a reliable mail server requires ongoing attention — monitoring deliverability, keeping software updated, checking blacklists, and tuning spam thresholds based on observed false positives and false negatives. But the result — a self-hosted, private, spam-filtered email service — is worth the effort for those who value control over their communications infrastructure.