Now that you’re monitoring the Linux operating system as well as the NTP server basics, it’s interesting to have a look at some more details about the DCF77 receiver. Honestly, there is only one more variable that gives a few details, namely the Clock Status Word and its Event Field. At least you have one more graph in your monitoring system. ;)
Wherever you’re running an NTP server: It is really interesting to see how many clients are using it. Either at home, in your company or worldwide at the NTP Pool Project. The problem is that ntp itself does not give you this answer of how many clients it serves. There are the “monstats” and “mrulist” queries but they are not reliable at all since they are not made for this. Hence I had to take another path in order to count NTP clients for my stratum 1 NTP servers. Let’s dig in:
Now that you have your own NTP servers up and running (such as some Raspberry Pis with external DCF77 or GPS times sources) you should monitor them appropriately, that is: at least their offset, jitter, and reach. From an operational/security perspective it is always good to have some historical graphs that show how any service behaves under normal circumstances to easily get an idea about a problem in case one occurs. With this post I am showing how to monitor your NTP servers for offset, jitter, reach, and traffic aka “NTP packets sent/received”.
During my work with a couple of NTP servers, I had many situations in which I just wanted to know whether an NTP server is up and running or not. For this purpose, I used two small Linux tools that fulfill almost the same: single CLI command while not actually updating any clock but only displaying the result. That is: ntpdate & sntp. Of course, the usage of IPv6 is mandatory as well as the possibility to test NTP authentication.
Yes, ScreenOS is end-of-everything (EoE), but for historical reasons I still have some of them in my lab. ;D They simply work, while having lots of features when it comes to IPv6 such as DHCPv6-PD. However, using IPv6-only NTP servers is beyond their possibilities. :(
Anyway, I tried using NTP authentication with legacy IP. Unfortunately, I had some issues with it. Not only that they don’t support SHA-1 but MD5, this MD5 key was also limited in its length to 16 characters. Strange, since ntp-keygen per default generates 20 ASCII characters per key. Let’s have a look:
I initially wanted to show how to use NTP authentication on a Pulse Connect Secure. Unfortunately, it does not support NTP over IPv6, which is mandatory for my lab. Ok, after I calmed down a bit, a configured it with legacy IP and got NTP authentication running. ;) Here’s how:
Configuring NTP authentication on the Infoblox Grid Master is quite simple. Everything is packed inside the single “NTP Grid Config” menu. You just have to enter the NTP keys respectively key IDs and enable authentication on the appropriate servers. In the case of incorrect authentication values an error message is logged. Very good, since this is not the case on some other network security devices (Palo, Forti).
Too bad that it only supports MD5 while SHA-1 should be used instead of.
A security device such as a firewall should rely on NTP authentication to overcome NTP spoofing attacks. Therefore I am using NTP authentication on the FortiGate as well. As always, this so-called next-generation firewall has a very limited GUI while you need to configure all details through the CLI. I hate it, but that’s the way Fortinet is doing it. Furthermore the “set authentication” command is hidden unless you’re downgrading to NTPv3 (?!?) and it only supports MD5 rather than SHA-1. Not that “next-generation”!
Finally, you have no chance of knowing whether NTP authentication is working or not. I intentionally misconfigured some of my NTP keys which didn’t change anything in the NTP synchronization process while it should not work at all. Fail!
Everyone uses NTP, that’s for sure. But are you using it with authentication on your own stratum 1 servers? You should since this is the only way to provide security against spoofed NTP packets, refer to Why should I run own NTP Servers?. As always, Palo Alto has implemented this security feature in a really easy way, since it requires just a few clicks on the GUI. (Which again is much better than other solutions, e.g., FortiGate, which requires cumbersome CLI commands.) However, monitoring the NTP servers, whether authentication was successful or not, isn’t implemented in a good way. Here we go:
This is how you can use NTP authentication on Cisco IOS in order to authenticate your external NTP servers respectively their NTP packets. Though it is not able to process SHA-1 but only MD5, you’re getting an authentic NTP connection. Let’s have a look:
Operating NTP in a secure manner requires the usage of NTP authentication, refer to my Why should I run own NTP Servers? blogpost. Using the Meinberg LANTIME NTP appliance with NTP authentication is quite simply since it requires just a few clicks. Even adding more and more keys (which requires manual work on any other Linux ntp installation) is done within clicks. That’s the way it should be.
As already pointed out in my NTP intro blogpost Why should I run own NTP Servers? it is crucial to leverage NTP authentication to have the highest trustworthiness of your time distribution all over your network. Hence the first step is to enable NTP authentication on your own stratum 1 NTP servers, in my case two Raspberry Pis with DCF77/GPS reference clocks.
When configuring a pool of NTP servers on a F5 BIG-IP load balancer you need to choose how to check if they are still up and running. There is no specific NTP monitor on a F5 BIG-IP that does an application layer health check (like there is for http or radius). The out-of-the-box options that can be used are only ICMP and UDP monitoring. Let’s first look at the pros and cons of using either (or both) of these monitors. Then let’s build a custom UDP monitor that does a better job at checking whether the NTP servers are still healthy.
As you hopefully already know, you should use at least three different NTP servers to get your time. However, there might be situations in which you can configure only one single NTP server, either via static IP addresses or via an FQDN. To overcome this single point of failure you can use an external load balancing server such as F5 LTM (in HA of course) to forward your NTP queries to one of many NTP servers. Here are some hints: