Pengcheng Laboratory

Sibilong Street Shenzhen Guangdong 518055 CN zhangb@pcl.ac.cn

Routing Computing-Aware Traffic Steering CATS Service Contact Instance SCI metric aggregation request dispatch health monitoring The Computing-Aware Traffic Steering (CATS) framework introduces the concept of a Service Contact Instance (SCI) as the client-facing entity responsible for receiving and dispatching service requests. While existing drafts define the framework and metrics, the concrete functional behavior of a Service Contact Instance remains underspecified. This document fills that gap by defining the functional implementation of a CATS Service Contact Instance. Specifically, it specifies how an SCI collects, aggregates, and reports service instance metrics to the CATS Service Metric Agent (C-SMA); how it monitors the health and status of underlying service instances; how it dispatches client requests to the most appropriate service instance based on local policy and real-time conditions; and how it maintains affinity and handles failure scenarios. This document complements and by providing the operational execution layer for the SCI.

The Computing-Aware Traffic Steering (CATS) framework defines a Service Contact Instance (SCI) as a client-facing function responsible for receiving requests in the context of a given service. The framework states that an SCI may dispatch service requests to one or more service instances and that steering beyond an SCI is hidden to both clients and CATS components. However, the framework does not specify: How an SCI collects and aggregates metrics from its underlying service instances. How an SCI monitors the health and availability of service instances. How an SCI dispatches incoming requests to the appropriate service instance. How an SCI reports metrics to the CATS Service Metric Agent (C-SMA). This document defines the functional implementation of a CATS Service Contact Instance to address these gaps. It specifies the internal architecture, metric collection mechanisms, health monitoring, request dispatch logic, and reporting interfaces required for an SCI to operate within the CATS framework. The SCI acts as the boundary between the CATS-aware network and the service site. It is responsible for: Collecting per-service-instance metrics (CPU, memory, GPU, throughput, queue depth, etc.). Aggregating these metrics into service-oriented abstractions (e.g., Global Available Slots, Computing Time) as defined in . Monitoring service instance health and adjusting reported metrics accordingly. Receiving client requests from the Egress CATS-Forwarder and dispatching them to the most suitable service instance. Reporting aggregated metrics and status to the C-SMA.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

This document makes use of the terms defined in and . In particular: CS-ID (CATS Service ID): An identifier for a service. CSCI-ID (CATS Service Contact Instance ID): An identifier for a service contact instance. In this document, it is interpreted operationally as a locator (e.g., IP address and port). Global Available Slots (GAS): The maximum number of concurrent requests a service site is willing and able to serve for a specific CS-ID at a given time. Computing Time: The time required for the site to perform one service request. C-SMA (CATS Service Metric Agent): The functional entity that collects service metrics and advertises them to C-PSes. Additionally, the following terms are used in this document: Service Instance (SI): A collection of running resources that are orchestrated following a service logic. An SCI may manage one or more SIs for the same CS-ID. Service Instance ID (SI-ID): A local identifier for a service instance within the scope of an SCI. Instance Metric: A raw or derived metric specific to a single service instance (e.g., instance CPU utilization, instance queue length). Health Status: A qualitative assessment of a service instance's operational state (e.g., healthy, degraded, failed).

An SCI is logically composed of the following internal functional components:

The internal components of an SCI are defined as follows: Responsible for collecting raw metrics from each service instance. This includes CPU utilization, memory usage, GPU utilization, request queue depth, response latency, throughput, and error rates. Responsible for aggregating instance-level metrics into service-oriented metrics (e.g., GAS, Computing Time) as defined in . The MA applies local policy and reference information (e.g., from ) to derive actionable metrics. Responsible for actively and passively monitoring the health of each service instance. It performs health checks, detects anomalies, and classifies instance status. Responsible for reacting to health changes. It updates the metric aggregator when instances fail or recover, and may trigger re-dispatch of in-flight requests. Responsible for receiving client requests from the Egress CATS-Forwarder and dispatching them to the most suitable service instance based on local load, health status, and affinity requirements. Responsible for maintaining session state, including affinity bindings between client flows and service instances. It tracks active sessions and notifies the Metric Aggregator of slot allocation and release. Responsible for formatting and sending aggregated metrics and status updates to the C-SMA.

The SCI sits at the boundary between the CATS network and the service site. Its relationships with other CATS components are: The SCI receives traffic from the Egress CATS-Forwarder. The SCI reports metrics to the C-SMA, which may be co-located with or adjacent to the SCI. The C-SMA advertises the SCI's metrics (along with the CSCI-ID) to the C-PS. The C-PS uses these metrics to make traffic steering decisions. The SCI does not directly interact with the C-PS or C-NMA; all control-plane communication goes through the C-SMA. The SCI is transparent to the client. The client sees only the CSCI-ID (e.g., an IP address and port) and is unaware of the internal service instances managed by the SCI.

The Metric Collector (MC) gathers the following categories of information from each service instance:

Raw computing resource utilization. Examples include: CPU utilization (%), Memory utilization (%), GPU utilization (%) and GPU memory usage, NPU/TPU utilization (if applicable), Disk I/O and network I/O rates. Service-level performance indicators. Examples include: Request queue depth, Average response time, Requests per second (throughput), Error rate (5xx errors, timeout rate). Operational state indicators. Examples include: Instance health (up/down), Load average, Number of active connections/sessions, Custom application-level status. The MC collects these metrics at a configurable sampling interval (e.g., every 5-10 seconds). The specific collection mechanism (e.g., Prometheus scraping, SNMP, gRPC, HTTP health endpoints) is deployment-specific and outside the scope of this document.

The MC SHOULD support multiple collection methods to accommodate different service instance types: The MC periodically queries each service instance via a standardized metrics endpoint (e.g., Prometheus /metrics, HTTP REST API, SNMP). This is suitable for instances that expose metrics via well-known interfaces. Service instances push metrics to the MC via a message queue or streaming protocol (e.g., gRPC streaming, MQTT, Kafka). This is suitable for high-frequency updates or event-driven metrics. A lightweight agent runs alongside each service instance and reports metrics to the MC. This is suitable for environments where instances cannot be modified to expose endpoints. The MC MUST support at least one of these methods. In mixed deployments, the MC MAY use different methods for different service instances.

The Metric Aggregator (MA) combines instance-level metrics into service-oriented metrics that are meaningful for CATS traffic steering. The key aggregated metrics are: The MA calculates the total GAS for the SCI by summing the available capacity of all healthy service instances. For each instance, the available capacity is derived from: the instance's maximum concurrent request capacity (it is the initial GAS value which can be calculated based on resource allocation and reference GAS information), the instance's current active session count (from the Session Manager), and the instance's health status (from the Health Monitor). Unhealthy instances contribute 0 to GAS. GAS = SUM_over_instances(max_capacity_i - active_sessions_i) for all healthy instances i The MA MAY apply a local policy factor (e.g., a safety margin of 80%) to prevent over-subscription. The MA estimates the Computing Time for the SCI based on the observed response times of service instances. This can be computed as: the weighted average of instance response times, weighted by instance load; the median or percentile (e.g., p95) of instance response times to account for outliers; or a dynamically adjusted estimate based on current load and historical trends. The MA also reports the min/max computing time and the associated input token counts for reference. The MA MAY also derive optional metrics such as Cost, Reputation, Security Label, and Capability (L1/L2) as defined in . The derivation algorithm is a local matter and is not standardized by this document. However, the MA MUST ensure that the reported metrics are consistent and comparable across updates.

The Health Monitor (HM) performs the following types of health checks on each service instance:

Verifies that the service instance process is running and responsive. This is typically done via a simple TCP connection or HTTP GET to a /health endpoint. Failure indicates the instance is down. Verifies that the service instance is ready to accept new requests. This may check: whether the instance has finished initialization, whether critical dependencies are available, and whether the instance is not in a maintenance mode. Verifies that the service instance is performing within acceptable bounds. This may check: response time against a threshold, error rate against a threshold, and resource utilization (e.g., CPU > 90% for 30 seconds). The HM SHOULD perform these checks at regular intervals (e.g., every 5-10 seconds for liveness, every 30 seconds for readiness and performance). The intervals and thresholds SHOULD be configurable.

The HM classifies each service instance into one of the following health states: HEALTHY: The instance is fully operational and accepting requests. DEGRADED: The instance is operational but experiencing performance issues (e.g., high latency, high error rate). New requests MAY be steered away, but existing sessions are maintained. UNHEALTHY: The instance is not operational or not ready. No new requests are dispatched to this instance. Existing sessions MAY be migrated or terminated based on policy. UNKNOWN: The HM cannot determine the instance's status (e.g., network partition). The instance is treated as UNHEALTHY until status is confirmed. State transitions trigger the following actions: HEALTHY -> DEGRADED: The HM notifies the MA to reduce the instance's contribution to GAS. The DP reduces or stops sending new requests to the instance. DEGRADED -> UNHEALTHY: The HM notifies the MA to set the instance's contribution to GAS to 0. The DP stops sending new requests. The SM initiates session migration or graceful termination for affected sessions. UNHEALTHY -> HEALTHY: The HM notifies the MA to restore the instance's contribution to GAS. The DP resumes sending requests. The HM MUST implement a hysteresis mechanism (e.g., require N consecutive failed checks before marking UNHEALTHY, and M consecutive passed checks before marking HEALTHY) to avoid flapping.

When the HM detects a health state change, the MA MUST adjust the aggregated metrics accordingly: When an instance becomes DEGRADED, the MA MAY reduce its max_capacity by a configured degradation factor (e.g., 50%) or set it to 0 based on local policy. When an instance becomes UNHEALTHY, the MA MUST set its contribution to GAS to 0. When an instance recovers to HEALTHY, the MA MUST restore its contribution to GAS based on current load. The Computing Time estimate MUST be recalculated to exclude UNHEALTHY instances and weight DEGRADED instances lower. These adjustments are reflected in the next metric report to the C-SMA. The MA SHOULD batch rapid health changes to avoid excessive updates.

The Dispatcher (DP) receives client requests from the Egress CATS-Forwarder and selects a service instance to handle each request. The dispatch decision is based on:

Affinity Requirements: If the request belongs to an existing session with affinity, the DP MUST dispatch to the same service instance (if healthy). Health Status: The DP MUST NOT dispatch to UNHEALTHY instances. It SHOULD avoid DEGRADED instances unless no HEALTHY instances are available. Load Balancing Policy: The DP selects among HEALTHY instances using a local load balancing algorithm. Supported algorithms include: Round-robin: Distribute requests evenly across instances. Least-connections: Send to the instance with the fewest active sessions. Weighted response time: Send to the instance with the lowest observed response time. Resource-aware: Send to the instance with the lowest resource utilization (CPU, memory, GPU). Slot-based: Send to the instance with the most available slots (max_capacity - active_sessions). Local Policy: The DP MAY apply additional policies such as cost optimization, security labels, or instance preference. The DP MUST handle the case where no HEALTHY instances are available. In this case, it MAY: Return an error to the client (e.g., HTTP 503 Service Unavailable). Queue the request and retry after a timeout. Dispatch to a DEGRADED instance as a last resort.

The Session Manager (SM) maintains affinity bindings between client flows and service instances. Affinity is identified by a flow key (e.g., 5-tuple: source IP, destination IP, source port, destination port, protocol). When a new request arrives: The SM checks if the flow key has an existing binding. If yes, and the bound instance is HEALTHY, the DP dispatches to that instance. If yes, but the bound instance is UNHEALTHY, the SM removes the binding and the DP selects a new instance. If no binding exists, the DP selects an instance and the SM creates a new binding. Affinity bindings have a configurable timeout. After the timeout expires with no activity, the SM removes the binding. The SM MUST support affinity at the flow level. It MAY also support affinity at the session level (e.g., for HTTP sessions identified by cookies or session IDs).

The SM tracks the lifecycle of each session: When a request is dispatched, the SM increments the active session count for the selected instance and records the flow binding. For long-lived sessions, the SM may refresh the binding timeout on each packet or keepalive. When the session ends (e.g., TCP FIN/RST, timeout, explicit logout), the SM decrements the active session count and removes the binding. The SM notifies the MA of session allocation and release events so that GAS can be updated. However, per-session changes do not necessarily trigger immediate reports to the C-SMA; the MA applies local aggregation and threshold policies.

The Reporting Interface (RI) communicates with the C-SMA to report aggregated metrics and status updates.

The RI SHOULD use a reliable transport (e.g., TCP, QUIC, or HTTP/2) to ensure metric delivery. The specific protocol is deployment-specific. In centralized deployments (e.g., SDN controller), the RI MAY use a RESTful API (e.g., RESTCONF ) or gRPC to push metrics to the C-SMA. In distributed deployments, the RI MAY use a routing protocol extension (e.g., BGP-LS , GRASP ) or a dedicated CATS metric distribution protocol.

The RI applies the following policies to control update frequency: A full report is sent at a fixed interval (e.g., every 60 seconds) to maintain soft state and prevent stale metrics. A delta report is sent when any metric crosses a configured threshold. Examples include: GAS changes by more than 10% or an absolute value of 50, Computing Time deviates by more than 20%, or Health status changes for any instance. A delta report is sent immediately upon critical events: all instances become UNHEALTHY (GAS = 0), an instance recovers from UNHEALTHY to HEALTHY, or security-related events (e.g., detected attack). The RI MUST implement rate limiting to prevent excessive updates during rapid fluctuations (e.g., during a thundering herd or DDoS attack). The specific thresholds and intervals SHOULD be configurable via the CATS Management Plane.

The report message contains the following fields: Report Message Fields

Field	Description
CSCI-ID	Identifier of the reporting SCI
CS-ID	Identifier of the service
Timestamp	Time of metric generation (Unix epoch)
Sequence Number	Monotonically increasing sequence number
GAS	Global Available Slots (uint32)
Computing Time	Estimated processing time in milliseconds (uint32)
Health Summary	Number of HEALTHY/DEGRADED/UNHEALTHY instances
Optional Metrics	Cost, Reputation, Security Label, Capability (if applicable)
Instance Details	Per-instance status (optional, for debugging)

The encoding details, including field lengths and wire format, are TBD and depend on the chosen transport protocol.

The SCI exposes a client-facing interface that is the CSCI-ID (e.g., an IP address and port). Clients send service requests to this interface, and the Egress CATS-Forwarder forwards traffic to it.

The SCI MUST support the following on the client-facing interface: Accept incoming connections/requests from the Egress CATS-Forwarder. Maintain transport-layer state (e.g., TCP connections) for the duration of the session. Support the service protocol (e.g., HTTP, gRPC, RTP) required by the CS-ID. The SCI MUST NOT expose internal service instance details (e.g., SI-IDs, internal IP addresses) to the client.

The SCI communicates with the Egress CATS-Forwarder via the underlay network. The Egress CATS-Forwarder is responsible for: Decapsulating CATS overlay traffic and forwarding it to the SCI's CSCI-ID. Receiving responses from the SCI and encapsulating them for return to the Ingress CATS-Forwarder. The SCI does not need to be CATS-aware; it operates as a standard service endpoint from the network perspective. However, the SCI MAY support CATS-specific signaling (e.g., for affinity or session state synchronization) if defined by future documents.

The SCI handles client requests and manages internal service instances, making it a critical security boundary. The following security measures are REQUIRED: The SCI MUST authenticate the Egress CATS-Forwarder to prevent unauthorized traffic injection. This may use mutual TLS, IPsec, or network-layer authentication. The SCI MUST verify that incoming requests are authorized for the requested CS-ID. This may involve validating tokens, certificates, or network-level policies. Service instances managed by the same SCI MUST be isolated from each other to prevent cross-tenant attacks. This includes network isolation (e.g., separate VLANs or namespaces) and resource isolation (e.g., CPU/memory limits). The RI MUST protect the integrity of metric reports sent to the C-SMA. This may use TLS, message authentication codes (MACs), or digital signatures. Metric reports SHOULD be encrypted to prevent disclosure of internal service topology and capacity. The SCI SHOULD implement rate limiting and connection throttling to protect against denial-of-service attacks that could exhaust GAS. Health check endpoints exposed by service instances SHOULD be protected to prevent spoofing or manipulation.

This document has no IANA actions at this time.

The authors thank the CATS working group for their valuable feedback and contributions to this document.