China Mobile

Beijing 100053 China yangjinwl@chinamobile.com

China Mobile

Beijing 100053 China tianyuchi@chinamobile.com

China Mobile

Beijing 100053 China chengweiqiang@chinamobile.com

Centec

Suzhou 215000 China wangjj@centec.com

Centec

Suzhou 215000 China zhanggy@centec.com

SEC IPSECME IPsec ESP ECMP LAG entropy load balancing When IPsec Encapsulating Security Payload (ESP) is used in tunnel mode, an intermediate node cannot inspect the encrypted inner headers to derive entropy for Equal-Cost Multipath (ECMP) or Link Aggregation Group (LAG) path selection. Path selection is then driven by outer header fields that are identical for every packet of a given tunnel, so all packets of the tunnel are placed on a single path regardless of the number of inner flows. This document defines the ESP Entropy Header, an IP protocol header that is placed immediately ahead of ESP and carries an entropy value in a fixed position that transit nodes can use for path selection. It also defines an Internet Key Exchange Protocol Version 2 (IKEv2) extension with which peers negotiate the use of the header on a per-Child-SA basis. The mechanism applies to both IPv4 and IPv6.

Introduction

Problem Statement IPsec provides security services for IP traffic at the network layer. ESP is the most widely deployed IPsec protocol and provides confidentiality, integrity, and anti-replay protection. In tunnel mode the original packet is encrypted and encapsulated in a new outer IP header. Intermediate nodes along the tunnel path perform ECMP and LAG path selection by computing a hash over selected packet header fields. For an ESP tunnel-mode packet, only the outer IP header is available to such a node. When a single tunnel carries multiple inner flows, the outer source and destination addresses are the same for every packet, so the hash yields the same result at each hop and all packets of the tunnel are assigned to the same path. The available paths are therefore not used evenly. This behavior is a recognized operational limitation of IPsec deployments. Comparable problems have been addressed in other encapsulations. MPLS uses entropy labels . IPv6 tunnels can use the flow label for the same purpose . For IPsec there is at present no header within the IP protocol stack, common to IPv4 and IPv6, that carries entropy in a fixed location for a transit node to use.

Overview This document defines an IP protocol header, the ESP Entropy Header, that immediately precedes the ESP header. The header carries a 16-bit Entropy field that is set by the IPsec sender and is available for inspection by transit nodes. A transit node that recognizes the assigned protocol number MAY include the Entropy field in its ECMP and LAG path-selection input. Use of the ESP Entropy Header is negotiated between IPsec peers using an IKEv2 Notify exchange, so that a peer includes the header only when its peer is prepared to process packets that carry it. Improving the entropy available for path selection, as this document does, is complementary to making the path-selection decision itself load-aware. The latter approach is described in ; the two mechanisms operate at different points and may be combined. The relationship is discussed in .

Design Considerations Entropy for encrypted tunnel traffic can be provided in more than one way. This section records the trade-offs that apply to the alternatives, so that the choice made in this document is documented rather than assumed. UDP encapsulation: and encapsulate ESP in UDP and use the UDP source port to carry entropy. This suits environments in which transit nodes already hash on the UDP five-tuple. It adds an 8-octet UDP header, requires a UDP checksum in IPv6 , and shares the UDP port space with NAT traversal , which can interact with the use of the source port for entropy. IPv6 flow label: uses the IPv6 flow label for ECMP and LAG in tunnels. It is native to IPv6 and adds no header, but is not available in IPv4 and provides a 20-bit field whose use for load balancing depends on transit support. Multiple Security Associations: distinct SPI values across several SAs can provide differentiation at the ESP layer. This increases IKE signaling and keying state and requires a transit node to inspect fields beyond the IP header. The ESP Entropy Header is defined as an IP protocol type and so applies to both IPv4 and IPv6. The Entropy field is at a fixed octet offset from the start of the header, which allows a transit node to locate it without parsing variable-length structures. This approach follows the precedent of Wrapped ESP , which defined an IP-protocol-numbered header that wraps ESP for a different purpose (traffic visibility); that precedent shows that an IP-protocol-numbered header adjacent to ESP is an established mechanism in the IPsec architecture.

Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Terminology This document uses the following terms.

IPsec Sender:

The IPsec peer that performs outbound ESP processing. In tunnel mode this is the tunnel ingress.

IPsec Receiver:

The IPsec peer that performs inbound ESP processing. In tunnel mode this is the tunnel egress.

Transit Node:

Any router or Layer 3 switch on the path between the IPsec sender and receiver that makes ECMP or LAG path selection decisions.

Entropy:

A value that, when included in the path-selection hash, differentiates packets that would otherwise hash to the same result, consistent with the use of the term in .

ESP Entropy Header Format The ESP Entropy Header is 8 octets in length. It is identified by its own IP protocol number (see ). In the IP header that precedes it, the Protocol field (IPv4) or the Next Header field (IPv6) carries this protocol number.

ESP Entropy Header

Next Header (8 bits):

Identifies the header that immediately follows the ESP Entropy Header, using a value from the IANA "Assigned Internet Protocol Numbers" registry. For the typical use with ESP this field is 50.

Hdr Len (8 bits):

Length of the ESP Entropy Header in 4-octet units, not including the first 4 octets. For the header defined in this document the value is 1, indicating a total length of 8 octets. This field allows a future extension to increase the header length without a new protocol number. A receiver that encounters a Hdr Len value it does not support MUST skip the indicated number of octets and continue processing at the Next Header.

Entropy (16 bits):

A 16-bit value. A transit node MAY use this field as input to ECMP and LAG path selection. The value is set by the IPsec sender. The means by which the sender selects the value is a local matter and is outside the scope of this document. To be useful for load distribution, the Entropy field has the following properties:

• All packets of the same inner flow MUST carry the same Entropy value, so that per-flow packet ordering is preserved across ECMP and LAG paths.
• The Entropy values used across the active flows of a tunnel SHOULD be approximately uniformly distributed.

Reserved (MBZ) (32 bits):

Set to zero on transmission and ignored on receipt. Reserved for future use.

Header Placement The ESP Entropy Header is placed between the outer IP header (and any IPv6 extension headers) and the ESP header. The following diagrams show the resulting packet structure.

IPv4 Tunnel Mode Packet Structure

IPv6 Tunnel Mode Packet Structure

The ESP Entropy Header is not covered by ESP encryption or integrity protection. It is visible in the clear to nodes on the path. The security consequences are discussed in .

Processing Requirements

IPsec Sender When the ESP Entropy Header has been negotiated for a Child SA (see ), the IPsec sender that chooses to include the header in an outbound packet on that SA does so subject to the following requirements.

• The outer IP Protocol (IPv4) or Next Header (IPv6) field MUST carry the protocol number assigned to the ESP Entropy Header.
• The Next Header field of the ESP Entropy Header MUST identify the following header, typically ESP (value 50).
• The Hdr Len field MUST be set to 1 for the header defined in this document.
• The Entropy field MUST carry a value that is the same for all packets of the same inner flow (see ).
• The Reserved field MUST be set to zero.
• ESP encryption and integrity protection are applied to the payload that follows the ESP Entropy Header, as in . The ESP Entropy Header is not part of the ESP-protected payload.

The IPsec sender MAY omit the ESP Entropy Header on some packets of an SA for which it has been negotiated, for example when no useful entropy can be derived for a packet. When the header is omitted, the outer IP Protocol or Next Header field MUST be set to the ESP value (50) and no ESP Entropy Header is present. A receiver MUST accept packets both with and without the header on an SA for which the header has been negotiated.

Transit Node A transit node that recognizes the ESP Entropy Header protocol number MAY include the Entropy field in the fields it uses for ECMP or LAG path selection. The Entropy field is at octets 2 and 3 of the header. A transit node MUST NOT modify any field of the ESP Entropy Header. A transit node that does not recognize the protocol number forwards the packet according to its existing behavior for the outer IP header; an unrecognized protocol number does not prevent forwarding. The mechanism is therefore incrementally deployable: a node that recognizes the header obtains improved path selection, and a node that does not continues to forward the traffic.

IPsec Receiver When the IPsec receiver has negotiated the ESP Entropy Header for a Child SA, it MUST accept and process an incoming packet whose outer IP Protocol or Next Header value indicates the ESP Entropy Header. Because the SPI that identifies the Child SA is carried in the ESP header, which follows the ESP Entropy Header, a receiver cannot determine the SA before it has parsed the ESP Entropy Header. A receiver that implements this specification therefore MUST process the ESP Entropy Header whenever the outer IP Protocol or Next Header value indicates it, and then continue to inbound ESP processing where the SA is resolved. Per-Child-SA negotiation () governs whether a sender is permitted to emit the header; it does not gate the receiver's ability to parse it. A node that does not implement this specification does not recognize the protocol number and handles the packet according to its existing behavior for an unknown IP protocol number, as described for transit nodes in .

• The receiver reads the Next Header field of the ESP Entropy Header to determine the following protocol. If the value is not recognized, the receiver MUST discard the packet and MAY log the event.
• The receiver uses the Hdr Len field to determine the length of the ESP Entropy Header and to locate the next header.
• The receiver does not examine the Entropy or Reserved fields; they are meaningful only to transit nodes.
• The receiver then processes the following header, typically ESP, according to normal inbound IPsec processing .

IKEv2 Negotiation Use of the ESP Entropy Header MUST be negotiated between peers before the header is sent. This document defines an IKEv2 Notify Message of type Status, ESP_ENTROPY_HEADER_SUPPORTED, for this purpose.

Negotiation Procedure The initiator includes the ESP_ENTROPY_HEADER_SUPPORTED notification in the IKE_AUTH or CREATE_CHILD_SA exchange that establishes or rekeys the Child SA. If the responder is willing to receive packets that carry the ESP Entropy Header on the Child SA, it includes the same notification in its response. The ESP Entropy Header is negotiated successfully when both the request and the response carry the ESP_ENTROPY_HEADER_SUPPORTED notification. If the response does not carry it, the initiator MUST NOT send packets that carry the ESP Entropy Header on the Child SA. Successful negotiation means that each peer is prepared to receive the header. Each peer decides independently whether to include the header in its own outbound packets; the notification does not require a peer to include the header in every packet.

Notify Payload Format

ESP_ENTROPY_HEADER_SUPPORTED Notify Payload

Protocol ID:

0 when the notification is sent in the IKE_AUTH exchange, or 3 (ESP) when it is sent in a CREATE_CHILD_SA exchange.

SPI Size:

0.

Notify Message Type:

ESP_ENTROPY_HEADER_SUPPORTED (value TBD2, see ).

This notification carries no data.

Path MTU Considerations The ESP Entropy Header adds 8 octets to each packet that carries it. An implementation that uses the header MUST account for the additional 8 octets when it performs Path MTU Discovery or when it determines the effective MTU for inner packets. The effective inner MTU when the header is in use is reduced by 8 octets. An implementation SHOULD use PMTUD and adjust the tunnel MTU accordingly. When PMTUD is not available, the implementation MUST use a conservative MTU estimate that accounts for the 8-octet header.

Security Considerations

Information Exposure The ESP Entropy Header is outside the ESP encryption and integrity boundaries, so the Entropy field is visible to observers on the path. The Entropy field does not directly carry inner packet content. Because its value is typically associated with an inner flow, an observer can infer that packets with the same Entropy value belong to the same flow, and over time the number of distinct Entropy values in a tunnel gives an estimate of the number of concurrent flows. This is a form of traffic analysis. An operator whose threat model includes flow-level traffic analysis by an on-path observer SHOULD weigh the benefit of improved load distribution against this exposure. Periodically changing the Entropy value of a long-lived flow reduces the duration over which packets can be correlated, at the cost of moving the flow between paths when the value changes.

Header Integrity The ESP Entropy Header is not integrity-protected by ESP. An on-path attacker that can modify packets could alter the Entropy field and influence transit path selection, for example to concentrate traffic on a subset of paths or to direct traffic onto a path the attacker observes. These risks are the same in kind as those from modification of other unprotected outer header fields such as the DSCP or the IPv6 flow label. An operator that requires integrity protection of the outer header chain MAY apply AH around the ESP Entropy Header.

Entropy Collisions The 16-bit Entropy field provides 65,536 values. When a tunnel carries many concurrent flows, two or more flows may share an Entropy value. A collision does not affect correctness; it reduces the granularity of load distribution for the affected flows.

Operational Considerations

Incremental Deployment A transit node that does not recognize the assigned protocol number forwards packets using its existing outer-header path selection, so the presence of the header on a path with non-supporting nodes does not cause packet loss. A benefit is obtained at each transit node that recognizes the header, independently of the other nodes on the path.

Coexistence with UDP Encapsulation When IPsec traffic is already encapsulated in UDP, for example for NAT traversal , the UDP source port can serve as entropy if the implementation varies it per flow. In that case the ESP Entropy Header adds no benefit and SHOULD NOT be negotiated.

Relationship to Load-Aware Path Selection The ESP Entropy Header improves the entropy that a transit node can use as input to a static path-selection hash. It does not change how a node reacts to load after a flow has been placed on a path. Load-aware path selection, such as the procedures of , adjusts placement in response to measured load and is complementary: better entropy improves the initial distribution, and load-aware selection corrects skew that develops afterward. The two mechanisms do not conflict.

IANA Considerations

IP Protocol Number IANA is requested to assign a value from the "Assigned Internet Protocol Numbers" registry. Allocation of an IP protocol number requires IESG approval per . Protocol Number Allocation

Decimal	Keyword	Protocol	Reference
TBD1	EEH	ESP Entropy Header	[This document]

IKEv2 Notify Message Type IANA is requested to assign a value from the "IKEv2 Notify Message Types - Status Types" registry. IKEv2 Notify Message Type Allocation

Value	Notify Message Type	Reference
TBD2	ESP_ENTROPY_HEADER_SUPPORTED	[This document]

References Normative References Informative References

Acknowledgements The use of an entropy value to distribute load for encapsulated traffic is established in the MPLS architecture through entropy labels and in IPv6 through the flow label . This document applies the same idea within the IPsec protocol stack.