It is not uncommon to see brute force SMTP authentication attacks against a mail server. Fail2Ban can be used to block the attackers source IP using iptables.

A sample log of an attack:

/var/log/auth.log

Jan 22 07:36:15 server saslauthd[2497]: pam_unix(smtp:auth): authentication failure; logname= uid=0 euid=0 tty= ruser= rhost= 
Jan 22 07:36:18 server saslauthd[2498]: pam_unix(smtp:auth): authentication failure; logname= uid=0 euid=0 tty= ruser= rhost= 
Jan 22 07:36:22 server saslauthd[2499]: pam_unix(smtp:auth): authentication failure; logname= uid=0 euid=0 tty= ruser= rhost= 
Jan 22 07:36:25 server saslauthd[2496]: pam_unix(smtp:auth): authentication failure; logname= uid=0 euid=0 tty= ruser= rhost= 

/var/log/mail.log

Jan 22 07:36:28 server sm-mta[1195]: o0LNaCc5001195: [119.129.227.125]: possible SMTP attack: command=AUTH, count=5

Fail2Ban is available in most Linux distributions. By default it does not block SMTP authentication attacks.

After installing the package in the distribution or downloading from the http://www.fail2ban.org website the following configuration is needed:

/etc/fail2ban/jail.local

[sendmail]

enabled  = true
port     = smtp,ssmtp
filter   = sendmail
logpath  = /var/log/mail.log
bantime  = 28800
action   = iptables-multiport[name=sendmail-auth, port="submission,465,smtp", protocol=tcp]
maxretry = 0

/etc/fail2ban/filter.d/sendmail.conf

# Fail2Ban configuration file
#
# Author: Jamie Morrison
#
# $Revision: 510 $
#

[Definition]

# Option:  failregex
# Notes.:  regex to match the password failures messages in the logfile. The
#          host must be matched by a group named "host". The tag "<HOST>" can
#          be used for standard IP/hostname matching and is only an alias for
#          (?:::f{4,6}:)?(?P<host>\S+)
# Values:  TEXT
#
failregex = \[<HOST>\]: possible SMTP attack: command=AUTH

# Option:  ignoreregex
# Notes.:  regex to ignore. If this regex matches, the line is ignored.
# Values:  TEXT
#
ignoreregex =