Sendmail 8.13.* and dkim-milter on RedHat 8.0

If you have the ability, please update your RedHat 8.0 system to a newer version. There are multiple vulnerabilities out for many different apps that exist in distro versions this old. Additionally, you will save yourself countless hours trying to make things work. You have been warned.
Summary: The version of sendmail that comes with RedHat 8.0 is 8.12.x. There is a milter API change between 8.12.x and 8.13.x and some new milters require this new API (this is the case for the dkim-milter). So the rpm build for sendmail-8.13.x requires a custom local openssl version. Even though the default location is /usr/local/ssl, I'm going to still specify that directory during the build process.
  1. Download, extract, compile, and install the latest openssl tarball. At the time of this writing, the latest version is 0.9.8e. This local version will not interfere with the default 0.9.6b version that every other application on your system uses. Do NOT uninstall the 0.9.6b version.
    wget http://www.openssl.org/source/openssl-0.9.8e.tar.gz tar -zxvf openssl-0.9.8e.tar.gz cd openssl-0.9.8e ./config --openssldir=/usr/local/ssl shared zlib no-idea no-mdc2 no-rc5 no-ec no-ecdh no-ecdsa no-krb5 make depend make all build-shared make rehash build-shared LD_LIBRARY_PATH="`pwd`"; export LD_LIBRARY_PATH; make -C test apps tests make install
  2. Build the new sendmail with this srpm. It is set for sendmail version 8.13.8, but any 8.13.* version should work if you get the sendmail tarball and put it in the SOURCES/ directory and modify the sendmail.spec file in the SPECS/ directory of your rpm build tree. Build it and install the resulting rpm. It will fail, but this is ok. The important thing here is to verify that the only thing it complains about is a missing libcrypto.so.0.9.8 and libssl.so.0.9.8. Yes, we did build those above, but the rpm database doesn't know about it. If this is the only thing rpm complains about, then install the sendmail rpms with the --nodeps option. This one will succeed.
    rpm -Fvh --nodeps /path/to/sendmail*.rpm
    If you prefer to build sendmail with source, here are the steps I used:
    tar -zxvf sendmail.8.13.8.tar.gz cd sendmail-8.13.8 cat > devtools/Site/site.config.m4 << EOF define(`confMAPDEF', `-DNEWDB -DMAP_REGEX -DSOCKETMAP -DNAMED_BIND=1') define(`confOPTIMIZE', ``-O2 -march=i586'') define(`confENVDEF', `-I/usr/include/db4 -I/usr/kerberos/include -Wall -DXDEBUG=0 -DTCPWRAPPERS -DNETINET6 -DHES_GETMAILHOST -DUSE_VENDOR_CF_PATH=1 -D_FFR_WORKAROUND_BROKEN_NAMESERVERS -D_FFR_SMTP_SSL') define(`confLIBDIRS', `-L/usr/kerberos/lib') define(`confLIBS', `-lnsl -lwrap -lhesiod -lcrypt -ldb') define(`confMANOWN', `root') define(`confMANGRP', `root') define(`confMANMODE', `644') define(`confMAN1SRC', `1') define(`confMAN5SRC', `5') define(`confMAN8SRC', `8') define(`confSTDIR', `/var/log/mail') define(`STATUS_FILE', `/var/log/mail/statistics') define(`confLIBSEARCH', `db resolv 44bsd') APPENDDEF(`conf_sendmail_ENVDEF', `-DSTARTTLS')dnl APPENDDEF(`conf_sendmail_LIBS', `-lssl -lcrypto')dnl APPENDDEF(`confLIBS', `-I/usr/local/ssl/include')dnl APPENDDEF(`confLIBDIRS', `-L/usr/local/ssl/lib -Xlinker -rpath -Xlinker /usr/local/ssl/lib')dnl APPENDDEF(`confENVDEF', `-DSASL=2 -D_FFR_UNSAFE_SASL')dnl APPENDDEF(`confLIBS', `-lsasl2 -lcrypto')dnl APPENDDEF(`conf_sendmail_ENVDEF', `-DMILTER')dnl APPENDDEF(`conf_libmilter_ENVDEF', `-D_FFR_MILTER_ROOT_UNSAFE')dnl EOF sh Build sh Build install
  3. Now we build the dkim-milter package. Extract the source:
    tar -zxvf dkim-filter-1.0.0.tar.gz cd dkim-filter-1.0.0
    There are two things to do to get it to compile. First we change a db lookup because for some reason, the version of db4 that comes with RH8.0 doesn't use a db_open() with the same amount of parameters as the source code expects.
    perl -pi -e 's/# if DB_VERSION_CHECK\(4,0,0\)/# if DB_VERSION_CHECK\(4,1,0\)/' dkim-filter/dkim-stats.c dkim-filter/stats.c
    The second thing we do is configure the m4 file:
    cat > devtools/Site/site.config.m4 << EOF APPENDDEF(`confENVDEF', `-D_FFR_STATS ') define(`confMANROOT', `/usr/share/man/man') EOF
    Now we build and install it:
    sh Build sh Build install mkdir /var/lib/dkim
    Put a config file in place:
    cat > /etc/sysconfig/dkim-milter << EOF DKIM_CONFIG="/etc/mail/dkim-milter.conf" EOF cat > /etc/mail/dkim.conf << EOF Background Yes Canonicalization simple Domain /etc/mail/dkim-milter.domains DNSTimeout 60 InternalHosts /etc/mail/dkim-milter.internalhosts # KeyFile /etc/mail/domainkeys/test.pem Mode v MTA MSA On-BadSignature accept On-DNSError accept On-InternalError accept On-NoSignature accept On-SignatureMissing accept Selector test Socket inet:10036@localhost Statistics /var/lib/dkim/test.db Syslog Yes X-Header No EOF cat > /etc/mail/dkim-milter.internalhosts << EOF 127.0.0.1 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 EOF hostname > /etc/mail/dkim-milter.domains
    Create an init script:
    cat > /etc/init.d/dkim-milter << EOF #! /bin/sh # # # dkim-filter Start/Stop the dkim milter daemon. # # chkconfig: 2345 85 15 # description: dkim is Domain Keys Identified Mail. It is a method of \ # computing cryptographic signatures for message headers in \ # an attempt to detect/prevent email forgeries. # processname: dkim-filter . /etc/init.d/functions if [ -f /etc/sysconfig/dkim-milter ]; then . /etc/sysconfig/dkim-milter else echo "No dkim-milter config file in /etc/sysconfig" exit fi case "$1" in start) echo -n "Starting DKIM milter: " daemon /usr/bin/dkim-filter -x $DKIM_CONFIG ;; stop) echo -n "Shutting down DKIM milter: " killproc dkim-filter ;; restart) $0 stop $0 start ;; reload|force-reload) echo -n "Reload service DKIM: " killproc dkim-filter -HUP ;; status) echo -n "Checking for service DKIM: " status /usr/bin/dkim-filter ;; *) echo "Usage: $0 {start|stop|status|try-restart|restart|reload}" exit 1 esac echo EOF
    Make the init script executable and set it to start at boot:
    chmod +x /etc/init.d/dkim-milter chkconfig --add dkim-milter
  4. Finally, let's start it all up:
    service dkim-milter start service sendmail restart
Edits: