I'd request that the built-in iptables on the CUCM, which we users can't adjust at all, could be autoadjusted by the CUCM itself to remove this DDOS vector, namely by restricting NTP to/from the CUCM only to these hosts:
why can that not be done?
I should add that this isn't merely a curiosity; one of my CUCMs has been targeted for this attack and there's very little I can do. "reconfigure your network so that our vulnerable appliance can't attack people because we evidently don't see fit to fix the vulnerability" isn't a very helpful 'solution'.
Are the attacks coming from outside your network? Restricting quieries to NTP services from outside the firewall should probably be blocked, anyway - or limited to a small set of servers. Other Linux systems are also subject to this amplification attack.
The attack is well described at
Are you being used as an amplifier, or are you a target of the packet floods?
thanks for your reply, Phillip.
yes, the attacks are coming from outside the network, and the CM is being used as an amplifier. and yes, while I can write ACLs in the network surrounding the CM to protect it, the CM itself does have iptables, darn it, and if it were allowed to be smarter could defend itself appropriately, which I argue Cisco should do.
Ah, I understand. I don't know why the CUCM does not use iptables in the device itself. I recommend that you also start a thread in the IP Telephony forum:
See if you can build a consensus among the group and have everyone ask for it in one loud voice. The best way to get a feature deployed to to have a lof of different customers ask for it.
good suggestion, thanks!
yes, there IS an iptables in the device itself, but it's pretty generically limited in what it can do.
Brent, Great observation and idea. The architecture of iptables inside the appliance doesn't really enable this approach. The iptables rules are selectively updated due to dependence on other processes. To that end UCM would only learn the IP of an endpoint when the endpoint registers. Every (un)registration event would then trigger a configuration change to iptables. This becomes unscalable very quickly. Generally on UCM all resources are dedicated to call processing as first priority and everything after that gets leftovers. This keeps processing of calls as highest priority. We are looking for opportunities to better leverage iptables and native capabilities in upcoming versions. Concurrently call processing remains priority #1. From a security perspective it is best practice to have an external device performing security, policing, and inspection. Regards, Wes
thanks, Wes--that response helps to frame the sometime-conflicting tensions between preserving performance and providing security.
I've been thinking about that, and the really excellent Cymru 'secure NTP template' (see
, trying to think about what could be done to offer better protection from the NTP attacks with less dynamicness, thinking that it's still important to offer something--all of my CUCMs that are outside firewalls have been attacked and participated in NTP-amplification attacks--and offer these suggestions as to things that the iptables might be leveraged to protect the CUCM, and at least as importantly everyone else FROM the CUCM, in a more static way:
* turn off control queries TO the CM--these are the vector into the CM that results in the amplification DDOS
* permit NTP into the CM only from the configured NTP servers the CM is using--yes, that's slightly 'dynamic', but will only occur infrequently and can be discretely done--scale is very small.
* the remaining really-dynamic part would be "only serve ntp to configured clients", and I can (reluctantly) understand why you push back on that. but if the first two points could be provided for, particularly the control-query filter which is the vector for at least the present threat, that's a huge improvement now.
the Cyrmu template under Unix NTP endsystems has some useful suggestions that could be adapted for CUCM iptables:
(quote from Cyrmu):
You can use your standard host firewall filtering capabilities to limit who the NTP process talks to. If you're using Linux and the host is acting as an NTP client only, the following iptables rules could be adapted to shield your NTP listener from unwanted remote hosts.
-A INPUT -s 0/0 -d 0/0 -p udp --source-port 123:123 -m state --state ESTABLISHED -j ACCEPT -A OUTPUT -s 0/0 -d 0/0 -p udp --destination-port 123:123 -m state --state NEW,ESTABLISHED -j
I didn't articulate it in the original note, but one simple fix to the CUCM's NTP implementation would be to fix it on all flavors of Cisco software to not respond to the monlist query that's the vector the following statement is from the excellent symantec discussion of the problem (http://www.symantec.com/connect/blogs/hackers-spend-christmas-break-launching-large-scale-ntp-reflection-attacks):
"How can you protect your servers? The easiest way to update to NTP version 4.2.7, which removes the monlist command entirely. If upgrading is not an option, you can start the NTP daemon with noquery enabled in the NTP conf file. This will disable access to mode 6 and 7 query packetts (which includes monlist).
"By disabling monlist, or upgrading so the the command is no longer there, not only are you protecting your network from unwanted reconnaissance, but you are also protecting your network from inadvertently being used in a DDoS attack."
THAT's what cisco ought to be doing. and as long as it is not doing that, I suggest that it can be considered a bug. and it's particularly problematic with appliances like the CUCM with captive OSs that are vulnerable, are attacking other systems, and can't defend themselves (unlike routers that have hooks to defend themselves). that needs to be fixed.
Brent, Interesting ideas indeed. Let me see what I can do. Updating NTP version across all VOS deployments (Communications Manager, Unity Connection, UCCX, CUP, telepresence, cisco prime, ...) is non-trivial. Disabling unnecessary functions is a central tenant of the appliance model. I'll look into it. -Wes
thanks for the encouragement, Wesley.
there's an even better page on this and mitigation strategies (aside from upgrading the version of ntpd to something vaguely recent--4.2.7p26 which is now about 4-5yrs old, i think--, which is the best thing and also avoids the NTP mode 7 DOS as well) on the ntp.org site: http://support.ntp.org/bin/view/Main/SecurityNotice#DRDoS_Amplification_Attack_using which describes, among other things, a change that can be made to the ntpd.conf file to disable monlist replies, which it seems would be a pretty easy short-term fix to the CUCM's immediate DDOS-amp problem and could be pushed out fairly easily by cisco. is there a possibility there?