In my last blogpost I showed how to perform a DNSSEC KSK rollover. I did it quite slowly and carefully. This time I am looking into an emergency rollover of the KSK. That is: What to do if your KSK is compromised and you must replace it IMMEDIATELY.

I am listing the procedures and commands I used to replace the KSK of my delegated subdomain dyn.weberdns.de with BIND. And, as you might already suggested, I am showing DNSViz graphs after every step since it greatly reveals the current DNSKEYs etc.

RFC 6781, “DNSSEC Operational Practices, Version 2”, describes all steps in case of a KSK compromise in section 4.2.1. There are two methods how to replace a compromised KSK, by either breaking the chain of trust or by keeping it intact. I decided to train the “Emergency Key Rollover Keeping the Chain of Trust Intact” version. This is basically the same as the normal “Double-Signature KSK rollover” described in section 4.1.2. That is:

1. introduce a new KSK (so that both KSKs sign the ZSK),
2. upload the new DS to the parent to replace (!) the DS record immediately,
3. wait until the TTL of the old DS record expires,
4. and remove the old KSK from your zone.

That is, there are two time critical things to keep in mind:

1. Since you must transfer the new DS record to your parent zone, you must wait until they publish it. If you can use an API, you’re lucky. If you must send it manually via mail and must wait for a human response, you’re not lucky…
2. You must wait until the TTL of the old DS record times out.

In my scenario I trained the emergency KSK rollover on one of my own delegated zones, dyn.weberdns.de. That is: I am controlling the actual zone as well as the parent zone. Good prerequisite for testing purposes. ;) Furthermore my TTL for the DS record was set to 3600 s = 1 h. Hence I was able to rollover (and delete) the compromised DNSKEY within one hour!

It was Friday evening, 2018-03-02. All times are noted as UTC.

19:33: Shit, my KSK for dyn.weberdns.de is compromised!

I was informed that the DNSKEY type 257 (KSK) for my very own zone was compromised. I immediately decided to do an emergency KSK rollover!

At this point DNSViz shows the single DS, a single KSK (grey, 37188) and a single ZSK (white, 58340) for the zone:

19:38: Generating a new KSK

I generated a new KSK for the zone (and corrected the ownership to be readable by BIND, as always):

Here you can see the new KSK with ID 45918, along with its dates in UTC+1:

After the creation I signed the zone again, which immediately uses the new key as well:

That is, querying the nameserver directly (and not a caching nameserver) revealed three DNSKEYs (2x KSK, 1x ZSK) along with its three RRSIGs over the DNSKEY resource record set:

DNSViz shows the new KSK (45918), while the DS from the parent zone still points to the old KSK (37188):

19:48: Replaced (!) the DS at the parent zone and scheduled the delete date of the compromised KSK

At first I generated the new DS resource record:

and replaced (!) it at the parent zone. Hence, querying the nameserver directly reveals only the new DS record:

DNSViz shows this single DS record pointing to the new KSK (45918) while the old and compromised KSK (37188) is still present at the zone:

At this point all new/initial DNS queries will get the new chain of trust, while cached entries all around the world will still use the old (cached) DS record. This is why you MUST NOT delete the compromised KSK until this cached DS record times out!

Now I set the “delete” date of the compromised key to be in one hour, which is the TTL of the DS resource record.

That is, looking at the key you can now see the delete time for 21:49:47 (UTC+1):
To inform BIND that this date has changed you must load the keys again:

20:50: BIND has deleted the compromised KSK

After this one hour of waiting, BIND successfully deleted the old (compromised) KSK out of the DNSKEY set. Hence, only one KSK and one ZSK, along with their two RRSIGs are present:

And of course DNSViz greatly shows that the single DS record points to the single KSK:

Conclusion

Since I was able to immediately replace the old DS record at my parents zone with the new DS record, I could delete the compromised KSK just a few moments after the TTL of the old DS timed out, which was about 1 hour in my example. The chain of trust was completely intact during this one hour period.

It is a best practice to train a KSK rollover *before* you must do it in a case of emergency. Write down the commands you need and schedule a KSK rollover on a regular basis to be prepared.

Featured image “Zusammenstoß mit Rettungswagen Mainz 10.11.17” by Wiesbaden112.de is licensed under CC BY-NC-ND 2.0.