IOAM network awareness for
Low Latency, Low Loss, and Scalable Throughput (L4S)Beijing Jiaotong
University3 Shangyuan Cun, Haidian DistrictBeijingP.R. Chinaweiquan@bjtu.edu.cnBeijing Jiaotong
University3 Shangyuan Cun, Haidian DistrictBeijingP.R. Chinawsu@bjtu.edu.cnBeijing Jiaotong
University3 Shangyuan Cun, Haidian DistrictBeijingP.R. China23111016@bjtu.edu.cnChina Telecom Research
InstituteSouth District of Future Science and Technology, Changping
DistrictBeijingP.R. Chinaliangxb2@chinatelecom.cnChina Telecom Research
InstituteSouth District of Future Science and Technology, Changping
DistrictBeijingP.R. Chinaliangjie6@chinatelecom.cnThis document specifies a framework that uses operational and
telemetry information collected by In Situ Operations, Administration,
and Maintenance (IOAM) to support the monitoring and parameter adjustment
of an L4S Dual-Queue Coupled AQM within a single limited domain. IOAM
Direct Export reports path and node information, such as hop-by-hop delay
and queue depth, to an IOAM Control Center (IOAM-C). The IOAM-C correlates
this information and may adjust operator-configurable AQM target or
threshold parameters within configured bounds. This document does not
modify the L4S ECN protocol, the IOAM data formats, or the packet
forwarding behavior defined by the referenced specifications.IntroductionThe L4S architecture uses Explicit
Congestion Notification (ECN) to provide early feedback about incipient
congestion. The L4S ECN protocol is defined in , and the Dual-Queue Coupled AQM framework is defined
in . ECN marking provides end-to-end congestion
indication, but it does not by itself identify the node that marked a
packet or expose per-hop queue state. In Situ Operations,
Administration, and Maintenance (IOAM) records and collects operational
and telemetry information while a packet traverses a path between two
points in a network. IOAM Direct Export is defined in .The goal of this framework is to make IOAM path and node information
available to an L4S deployment within the same limited domain. An IOAM
Control Center (IOAM-C) stores, correlates, and analyzes the exported
information and provides bounded parameter updates to operator-managed
Dual-Queue Coupled AQM instances.This specification defines how data collected by IOAM can provide
path visibility and bounded parameter input to a Low Latency, Low Loss,
and Scalable Throughput (L4S) deployment. The terms "encapsulation" and
"decapsulation" are used in this document in the same way as . An IOAM
encapsulating node incorporates one or more IOAM Option-Types into
packets that IOAM is enabled for.TerminologyThe key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP 14
when, and only when,
they appear in all capitals, as shown here.L4S: Low Latency, Low Loss, and Scalable Throughput (L4S) as defined
in .L4S Dual-Queue Coupled AQM: A framework for coupling the Active Queue
Management (AQM) algorithms in two queues intended for flows with
different responses to congestion.IOAM: In situ Operations, Administration, and Maintenance as defined
in .OAM: Operations, Administration, and Maintenance.IOAM Transit Node (IOAM-T): An IOAM transit node updates one or more
of the IOAM-Data-Fields. If both the Pre-allocated and the Incremental
Trace Option-Types are present in the packet, each IOAM transit node
will update at most one of these Option-Types.IOAM Encapsulating Node (IOAM-E): An IOAM encapsulating node
incorporates one or more IOAM Option-Types into packets that IOAM is
enabled for. If IOAM is enabled for a selected subset of the traffic,
the IOAM encapsulating node is responsible for applying the IOAM
functionality to the selected subset.IOAM Decapsulating Node (IOAM-DE): An IOAM decapsulating node removes
any IOAM Option-Types from packets and processes and/or exports the
associated data.IOAM Node Identifier: The combination of IOAM node_id and IOAM
Namespace-ID identifies an IOAM node within the operational context of
an IOAM domain.Direct Export: Direct Export is an IOAM mode of operation within
which IOAM data are to be directly exported to a collector rather than
be collected within the data packets. The IOAM Direct Export Option-Type
consists of a fixed-size "IOAM direct export option header". Direct
Export for IOAM is defined in .IOAM network awareness in L4S frameworkThe following are the system components of the L4S-IOAM
framework.L4S-IOAM
Schematic +--------------+ +-----------+
+-------------+ | | L4S |
| Monitoring/ | |======>| DualQ AQM |
| Analytics | | | |
| IOAM-C |-+ +-----------+
+-------------+
^
| Processed/interpreted/
| aggregated IOAM data
|
+--------------+
+-------------+ |
| IOAM data | |
| processing | |
| system(s) |-+
+-------------+
^
| Raw export of
| IOAM data
|
+--------------+-------+------+--------------+
| | | |
| | | |
User +---+----+ +---+----+ +---+----+ +---+-----+
packets | IOAM-E | | IOAM-T | | IOAM-T | | IOAM-DE |
-------->| Node |====>| Node |====>| Node |====>| Node |-->
| | | A | | B | | |
+--------+ +--------+ +--------+ +---------+IOAM Control Center (IOAM-C): The IOAM-C stores and manages exported
network information and makes decisions based on an analysis of this
information. Its inputs include hop-by-hop delay, queue depth, and node
and interface identifiers. Its outputs are limited to
operator-configurable target and threshold parameters of the Coupled AQM
at IOAM-capable nodes, and each output MUST remain within
operator-configured bounds. The IOAM-C MUST NOT change L4S ECN codepoint
processing. If telemetry is unavailable, stale, or inconsistent, the
IOAM-C MUST NOT issue a parameter update based on that telemetry.IOAM Encapsulating Node (IOAM-E): The IOAM-E selects the traffic to
which IOAM is applied and incorporates the selected IOAM Option-Types
into packets, as specified in and .IOAM-E, IOAM-T, and IOAM-DE nodes that support Direct Export send
selected IOAM data to a data processing system or to the IOAM-C using an
operator-selected export mechanism. Before the information is used by
the IOAM-C, records from individual nodes are correlated using available
Namespace-ID, Flow ID, and Sequence Number information.Figure 2 summarizes the L4S forwarding and queueing functions
monitored by this framework.L4S Forwarding
and Queueing Functions
(3) (2) (4)
.-------^------..------------^------------------.
_________
,-(1)-----. _____ | IOAM |
; ________ : L4S -------. | | /_ _ _| Monitor |
:|Scalable| : _ __\ ||__\_|mark | \ | |_________|
:| sender | : __________ / / || / |_____| \ | _________
:|________|\; | |/ -------' ^ \ | |condit'nl|
`---------'\_| IP-ECN | Coupling : \_|_|priority |_\
________ / |Classifier| : / | |scheduler| /
|Classic |/ |__________|\ -------. __:__ / | |_________|
| sender | \_ __\ || | ||__\_|mark/|/ |
|________| / || | || / |drop |/_ _|
Classic -------' |_____|\
(1) Scalable sending host
(2) Isolation in separate network queues
(3) Packet identification protocol
(4) Monitor in network queuesInformation DetailsIOAM for L4S is used to enhance diagnostics of L4S networks. It
complements the Explicit Congestion Notification (ECN) protocol for L4S
described in .Figure 3 shows examples of information used to provide path and node
awareness for an L4S deployment.L4S-IOAM
Information Details+-------------+----------------------+
| Awareness | Network |
| information | information |
+-------------+----------------------+
| | IOAM node location; |
| | IOAM node type; |
| Capability | IOAM node ID; |
| parameters | Topology information.|
| | |
| | |
| | |
| | |
| | |
+-------------+----------------------+
| | Service request |
| Status | information; |
| parameters | Traffic features; |
| | Communication |
| | information. |
+-------------+----------------------+"IOAM tracing data" is expected to be
collected at every IOAM transit node that a packet traverses to ensure
visibility into the entire path that a packet takes within an
IOAM-Domain. In other words, in a typical deployment, all nodes in an
IOAM-Domain would participate in IOAM and, thus, be IOAM transit nodes,
IOAM encapsulating nodes, or IOAM decapsulating nodes. If not all nodes
within a domain are IOAM capable, IOAM tracing information (i.e., node
data, see below) will only be collected on those nodes that are IOAM
capable. IOAM tracing can, for example, collect the following types of
information:
Identification of the IOAM node. An IOAM node
identifier can match to a device identifier or a particular control
point or subsystem within a device.
Identification of the interface that a packet
was received on, i.e., ingress interface.
Identification of the interface that a packet
was sent out on, i.e., egress interface.
Time of day when the packet was processed by
the node as well as the transit delay. Different definitions of
processing time are feasible and expected, though it is important that
all devices of an IOAM-Domain follow the same definition.
Generic data, which is format-free
information, where the syntax and semantics of the information are
defined by the operator in a specific deployment. For a specific
IOAM-Namespace, all IOAM nodes should interpret the generic data the
same way. Examples for generic IOAM data include geolocation
information (location of the node at the time the packet was
processed), buffer queue fill level or cache fill level at the time
the packet was processed, or even a battery charge level.
Information to detect whether IOAM trace data
was added at every hop or whether certain hops in the domain weren't
IOAM transit nodes.
Data that relates to how the packet traversed
a node (transit delay, buffer occupancy in case the packet was
buffered, and queue depth in case the packet was queued).
L4S-IOAM Direct Export Mode
Export of Export of Export of Export of
IOAM data IOAM data IOAM data IOAM data
^ ^ ^ ^
| | | |
| | | |
User +---+----+ +---+----+ +---+----+ +---+-----+
packets | IOAM-E | | IOAM-T | | IOAM-T | | IOAM-DE |
-------->| Node |====>| Node |====>| Node |====>| Node |-->
| | | A | | B | | |
+--------+ +--------+ +--------+ +---------+
This framework uses Direct Export mode for L4S-IOAM information
gathering. OAM information about each IOAM node a packet
traverses is collected and immediately exported to a collector.
Direct Export could be used in situations where per-hop tracing
information is desired, but gathering the information within the
packet -- as with per-hop tracing -- is not feasible. Rather than
automatically correlating the per-hop tracing information, as done
with per-hop tracing, Direct Export requires a collector to
correlate the information from the individual nodes. In addition,
all nodes enabled for Direct Export need to be capable of
exporting the IOAM information to the collector.As required by , a node that performs
Direct Export MUST support export rate limiting. When records from
multiple nodes need to be correlated, the Flow ID and Sequence Number
fields defined by SHOULD be used.This information can help the IOAM-C identify the location and
severity of congestion. This document does not define new per-flow
balancing, scheduling, or ECN codepoint behavior.Use CasesThis section identifies representative environments in which IOAM
path visibility can complement an L4S deployment. In each case, IOAM is
used within a limited domain to locate and characterize congestion; the
L4S service continues to operate according to ,
, and .Where a use case spans multiple administrations, each IOAM deployment
and each AQM parameter update remain confined to the limited domain
controlled by one administration.The following use cases for L4S are being considered by various
interested parties:
where the bottleneck is one of various types of access network,
e.g., DSL, Passive Optical Networks (PONs), DOCSIS cable,
mobile, satellite; or where it's a Wi-Fi link
private networks of heterogeneous data centres, where there is
no single administrator that can arrange for all the simultaneous
changes to senders, receivers, and networks needed to deploy
DCTCP:
a set of private data centres interconnected over a wide
area with separate administrations but within the same
company
a set of data centres operated by separate companies
interconnected by a community of interest network
(e.g., for the finance sector)
multi-tenant (cloud) data centres where tenants choose
their operating system stack (Infrastructure as a Service
(IaaS))
different types of transport (or application) congestion
control:
elastic (TCP/SCTP);
real-time (RTP, RMCAT); and
query-response (DNS/LDAP).
where low delay QoS is required but without
inspecting or intervening above the IP layer :
Mobile and other networks have tended to inspect higher
layers in order to guess application QoS requirements.
However, with growing demand for support of privacy and
encryption, L4S offers an alternative. There is no need to
select which traffic to favour for queuing when L4S can give
favourable queuing to all traffic.
If queuing delay is minimized, applications with a fixed delay
budget can communicate over longer distances or via more circuitous paths, e.g., longer
chains of service functions or of onion
routers.
If delay jitter is minimized, it is possible to reduce the
dejitter buffers on the receiving end of video streaming, which
should improve the interactive experience.
ContributorsThanks to Xue Zhang, Ziheng Xu and Xuetong Hu for their contributions to this document.IANA ConsiderationsNone.Security ConsiderationsThe security considerations of IOAM and IOAM Direct Export described
in and apply. This
framework is intended for use within a limited domain. An operator MUST
prevent IOAM data and control messages from leaking outside the domain
and MUST reject telemetry from unauthorized sources or namespaces.Telemetry export and AQM parameter update interfaces MUST be
authenticated, integrity-protected, and restricted to authorized
entities. The IOAM-C MUST validate the source, namespace, freshness, and
consistency of telemetry before using it. Stale, replayed, malformed, or
inconsistent telemetry MUST NOT cause an AQM parameter update.Direct Export consumes processing and network resources and can be
abused to create excessive export traffic. Exporting nodes MUST apply
the rate-limiting requirements of . If telemetry
export or the IOAM-C becomes unavailable, packet forwarding and the
existing local AQM configuration MUST continue without dependence on
the unavailable component.ReferencesNormative ReferencesInformative ReferencesAcknowledgmentsThe authors wish to thank , and , for their
invaluable comments.