Some years ago I wrote a blog post called “Basic syslog-ng Installation“. While I used it myself quite often in my labs or at the customers’ sites, it shows only basic UDP transport which is both unreliable and insecure. So, let’s have a look at a fresh installation of syslog-ng with TLS support for security reasons. However, TCP and UDP as transport are covered as well for the support of legacy systems.

Prerequisite: X.509 Certificate

The secure transport of log messages relies on a well-known TLS connection. Therefore, the server needs a valid X.509 certificate, commonly known as a TLS certificate or the like. Of course, the server needs the certificate plus its corresponding private key. ;) If you are an enterprise, you can generate/sign a certificate as always within your PKI. If you don’t have your own PKI but still want to use a trusted certificate, you can use Let’s Encrypt, e.g., with the acme.sh script. (Out of scope for this blog post.)

The easiest way is to generate a self-signed certificate for this use case:

• use the FQDN of the syslog server as the common name
• the subject alternative names (SAN) should contain the FQDN as well, and additionally the IP addresses of the server (if your syslog clients use the IP address of the server rather than the FQDN, which is likely)
• though modern browsers only accept certificates that have a lifetime < 1 year, I don’t see this requirement here. Hence I am using 3 years as the lifetime.

TL;DR: Use the following OpenSSL command to generate your certificate. Replace the FQDN and the IP addresses according to your needs:

You’ll have two files: syslog.crt and syslog.key. Optionally, you can verify that the .crt file starts with —–BEGIN CERTIFICATE—– while the .key file starts with —–BEGIN PRIVATE KEY—–.

Installation

I am using a Ubuntu 20.04.2 LTS version for this guide. It all starts with:

On my fresh Ubuntu, this installed 33 new packages. After the installation, the syslog-ng daemon is already running, though not listening on any interfaces:

Use  syslog-ng -V to see the installed version. In my case, it’s 3.25.1.

Configuration

Concerning the two certificate files: Change the ownership to root:root, make a folder for them within the syslog-ng folder and move them to that place:

For the following TLS related config snippets, I used the “syslog-ng Open Source Edition 3.30 – Administration Guide“.

I am using a single configuration file within the “conf.d” subfolder:

Using the following content, the service will listen for IPv6 and IPv4 on UDP (514), TCP (514) and TLS (6514), while creating a new folder for each logging source, year & month, writing into a new file every day. Note that I am using the “syslog” driver for receiving IETF/RFC5424 messages rather than the old “network” driver for merely BSD/RFC3164:

Restarting the server:

Verifying that the service is listening on those three ports:

Basically, that’s it. ;) You can now point your clients to this syslog-ng server. When the very first syslog messages are received, the /var/log/syslog-ng and the corresponding subfolders will be created. For example, my syslog client with IP address 2001:470:7250::1 sending logs in June 2021 logs into: /var/log/syslog-ng/2001:470:7250::1/2021/06.

Please note that logs coming through legacy IP connections are stored in the following way: ::ffff:<IPv4-address>. Have a look at this blog post for more details. Example:

TCP Port?!?

One question that took me some time: Which port is used for syslog-over-TCP? Normally, the same port as for UDP should fit, that is: 514. But the TCP port 514 is *not* registered for “syslog” but for “shell”, ref: IANA. This is unlike other common protocols such as DNS, where port 53 is registered for UDP and TCP. Uhm. RFC 1395 lists TCP port 601 for reliable syslog connections, which is listed at the IANA as well. However, RFC 6587 tells this:

TL;DR: Decide whatever you want, but maybe 514 is a good idea because everyone recognizes it as syslog. ;) A Twitter poll from me gave the same result:

Hey network admins: Which port are you using for Syslog over TCP? (There's no well-known port for it…)

— Johannes Weber ? (@webernetz) June 7, 2021

Verification/Troubleshooting

You have multiple ways to troubleshoot the server and especially its TLS configuration. For example, you can use tcpdump to check if connections to ports 514 or 6514 are coming in at all (replace “ens160” with the name of your NIC):

Example connection on TCP 514:

Example connection on TLS via TCP 6514:

To check for the open ports, use Nmap from a different machine:

Examples for IPv6 and legacy IP:

It’s correct that UDP 514 is “open|filtered” and UDP 6514 is closed.

To verify the X.509 certificate you can use “openssl s_client”:

In my case: (It’s using TLSv1.3 as you can see.)

And to enumerate the provided TLS ciphers, use Nmap again. However, please note that Nmap does not support TLSv1.3 at the moment (Nmap 7.91):

My example:

A Little Capture

And by the way: This is what it looks like in Wireshark. Standard TLS handshake, that is: client hello, server hello, certificate, server key exchange, client key exchange, and so on, finally followed by (unreadable since encrypted) application data:

You can find these captured packets in my most recent Ultimate PCAP.

All the best! God bless you.

Featured image “journals” by Meagan is licensed under CC BY-NC 2.0.