Basic concepts
Security protocols
IPsec comes with two security protocols:
AH (protocol 51)—Provides data origin authentication, data integrity, and anti-replay services by adding an AH header to each IP packet. AH is suitable only for transmitting non-critical data because it cannot prevent eavesdropping, although it can prevent data tampering. AH supports authentication algorithms such as Message Digest (MD5) and Secure Hash Algorithm (SHA-1).
ESP (protocol 50)—Provides data encryption as well as data origin authentication, data integrity, and anti-replay services by inserting an ESP header and an ESP trailer in IP packets. Unlike AH, ESP encrypts data before encapsulating the data to guarantee data confidentiality. ESP supports encryption algorithms such as Data Encryption Standard (DES), 3DES, and Advanced Encryption Standard (AES), and authentication algorithms such as MD5 and SHA-1. The authentication function is optional to ESP.
Both AH and ESP provide authentication services, but the authentication service provided by AH is stronger. In practice, you can choose either or both security protocols. When both AH and ESP are used, an IP packet is encapsulated first by ESP and then by AH. Figure 74 shows the format of IPsec packets.
Security association
A security association (SA) is an agreement negotiated between two communicating parties called IPsec peers. It comprises a set of parameters for data protection, including security protocols, encapsulation mode, authentication and encryption algorithms, and shared keys and their lifetime. SAs can be set up manually or through IKE.
An SA is unidirectional. At least two SAs are needed to protect data flows in a bidirectional communication. If two peers want to use both AH and ESP to protect data flows between them, they construct an independent SA for each protocol.
An SA is uniquely identified by a triplet, which consists of the security parameter index (SPI), destination IP address, and security protocol identifier (AH or ESP).
An SPI is a 32-bit number for uniquely identifying an SA. It is transmitted in the AH/ESP header. A manually configured SA requires an SPI to be specified manually for it; an IKE created SA will have an SPI generated at random.
A manually configured SA never ages out. An IKE created SA has a specified period of lifetime, which comes in two types:
Time-based lifetime, which defines how long the SA can be valid after it is created.
Traffic-based lifetime, which defines the maximum traffic that the SA can process.
The SA becomes invalid when either of the lifetime timers expires. Before the SA expires, IKE negotiates a new SA, which takes over immediately after its creation.
Encapsulation modes
IPsec supports the following IP packet encapsulation modes:
Tunnel mode—IPsec protects the entire IP packet, including both the IP header and the payload. It uses the entire IP packet to calculate an AH or ESP header, and then encapsulates the original IP packet and the AH or ESP header with a new IP header. If you use ESP, an ESP trailer is also encapsulated. Tunnel mode is typically used for protecting gateway-to-gateway communications.
Transport mode—IPsec protects only the IP payload. It uses only the IP payload to calculate the AH or ESP header, and inserts the calculated header between the original IP header and payload. If you use ESP, an ESP trailer is also encapsulated. The transport mode is typically used for protecting host-to-host or host-to-gateway communications.
Figure 74 shows how the security protocols encapsulate an IP packet in different encapsulation modes. Data represents the transport layer data.
Figure 74: Encapsulation by security protocols in different modes
Authentication algorithms and encryption algorithms
Authentication algorithms
IPsec uses hash algorithms to perform authentication. A hash algorithm produces a fixed-length digest for an arbitrary-length message. IPsec peers respectively calculate message digests for each packet. If the resulting digests are identical, the packet is considered intact.
IPsec supports the following hash algorithms for authentication:
MD5, which takes a message of arbitrary length as input and produces a 128-bit message digest.
SHA-1, which takes a message of a maximum length less than the 64th power of 2 in bits as input and produces a 160-bit message digest.
Compared with SHA-1, MD5 is faster but less secure.
Encryption algorithms
IPsec mainly uses symmetric encryption algorithms, which encrypt and decrypt data by using the same keys. The following encryption algorithms are available for IPsec on the device:
Data Encryption Standard (DES), which encrypts a 64-bit plain text block with a 56-bit key. DES is the least secure but the fastest algorithm. It is sufficient for general security requirements.
Triple DES (3DES), which encrypts plain text data with three 56-bit DES keys. The key length totals up to 168 bits. It provides moderate security strength and is slower than DES.
Advanced Encryption Standard (AES), which encrypts plain text data with a 128-bit, 192-bit, or 256-bit key. AES provides the highest security strength and is slower than 3DES.
IPsec SA setup modes
There are two IPsec SA setup modes:
Manual mode. In this mode, you manually configure and maintain all SA settings. Advanced features like periodical key update are not available. However, this mode implements IPsec independently of IKE.
ISAKMP mode. In this mode, IKE automatically negotiates and maintains IPsec SAs for IPsec.
If the number of IPsec tunnels in your network is small, use the manual mode. If the number of IPsec tunnels is large, use the ISAKMP mode.
IPsec tunnel
An IPsec tunnel is a bidirectional channel created between two peers. An IPsec tunnel comprises one or more pairs of SAs.