Formal validation of the security properties of AMT's three-way handshake

Abstract

Formal Validation of Security Properties of AMT's Three-way Handshake Ali SalemMulticasting is a technique for transmitting the same information to multiple receivers over IP networks. It is often deployed on streaming media applications over the Internet and private networks. The biggest problem multicast introduces today is that it is an "all or nothing" solution. Every element on the path between the source and the receivers (links, routers, firewalls) requires multicast protocols to be enabled. Furthermore, multicast has a conceptual business model, and therefore is not an easy case to make. These factors, embedded deep in technology, but ultimately shaped by economics, led to a lack of multicast deployment. To address this problem, the AMT (Automatic IP Multicast without explicit Tunnels) specification has been developed by the Network Working Group at the IETF. This specification is designed to provide a mechanism for a migration path to a fully multicast-enabled backbone.It allows multicast to reach unicast-only receivers without the need for any explicit tunnels between the receiver and the source. We have formally validated the three-way handshake in the AMT specification using AVISPA against two main security goals: secrecy and authentication. We have demonstrated that the authentication goal is not met: an attacker can masquerade as an AMT relay, and the AMT gateway (at the end user) cannot distinguish a valid relay from an invalid one. Another attack was also found where an intruder can disconnect or shutdown a valid session for a valid end-user using a replay attack.