Network Working Group A. Artamokhov Internet-Draft Independent Intended status: Experimental 30 October 2026 Expires: 3 May 2027 Federation Between Capability Indexes: Crawl, Reciprocal Identification and Admission draft-artamokhov-dawn-index-federation-01 Abstract This document describes how independently operated capability indexes discover each other by following discovery documents, how a reading index identifies itself to the index it reads, and an admission model that distinguishes a peer that is known from a peer whose catalogue is indexed. It also describes how an index can preview a peer's catalogue before admitting it. Payment, identity, reputation and invocation are out of scope. The document is intended as input to the DAWN gap analysis on federation between directories. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on 3 May 2027. Copyright Notice Copyright (c) 2026 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/ license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components Artamokhov Expires 3 May 2027 [Page 1] Internet-Draft Index Federation October 2026 extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 1.1. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.2. Requirements Language . . . . . . . . . . . . . . . . . . 3 1.3. Terminology . . . . . . . . . . . . . . . . . . . . . . . 3 2. Crawl protocol . . . . . . . . . . . . . . . . . . . . . . . 4 3. Reciprocal identification . . . . . . . . . . . . . . . . . . 4 4. Announcement . . . . . . . . . . . . . . . . . . . . . . . . 5 5. Admission . . . . . . . . . . . . . . . . . . . . . . . . . . 6 5.1. States . . . . . . . . . . . . . . . . . . . . . . . . . 6 5.2. Open and closed admission . . . . . . . . . . . . . . . . 6 5.3. Rules . . . . . . . . . . . . . . . . . . . . . . . . . . 7 5.4. Automatic assessment . . . . . . . . . . . . . . . . . . 7 5.5. Catalogue preview . . . . . . . . . . . . . . . . . . . . 7 6. Relationship to other work . . . . . . . . . . . . . . . . . 8 7. Security Considerations . . . . . . . . . . . . . . . . . . . 8 7.1. Request forgery . . . . . . . . . . . . . . . . . . . . . 8 7.2. Key pinning . . . . . . . . . . . . . . . . . . . . . . . 9 7.3. Resource exhaustion . . . . . . . . . . . . . . . . . . . 9 7.4. Split views . . . . . . . . . . . . . . . . . . . . . . . 9 7.5. Privacy . . . . . . . . . . . . . . . . . . . . . . . . . 9 7.6. Limits . . . . . . . . . . . . . . . . . . . . . . . . . 9 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 10 8.1. HTTP field name registration . . . . . . . . . . . . . . 10 9. Implementation Status . . . . . . . . . . . . . . . . . . . . 10 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 10 10.1. Normative References . . . . . . . . . . . . . . . . . . 10 10.2. Informative References . . . . . . . . . . . . . . . . . 11 Implementation experience . . . . . . . . . . . . . . . . . . . . 12 Changes from -00 . . . . . . . . . . . . . . . . . . . . . . . . 13 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . 13 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 13 1. Introduction Capability indexes are operated by many independent parties, and each lists only what its operator has seen. A client sees only the catalogue of the index it queries, and a provider has to register with each index separately. [I-D.jimenez-dawn-discovery-landscape] (Section 6.2) identifies the lack of an interoperable federation mechanism between directories as a gap. Existing designs federate through a shared distributed hash Artamokhov Expires 3 May 2027 [Page 2] Internet-Draft Index Federation October 2026 table [I-D.mp-agntcy-ads], through configured trust relationships [I-D.pioli-agent-discovery], or not at all ([A2A] leaves registries to deployments; [MCPREG] has one upstream registry with mirrors). This document describes a simpler arrangement. Each index is an HTTPS server that publishes a discovery document [DISCOVERY]. Indexes find each other by following those documents, a reading index identifies itself to the index it reads, and each index decides per peer whether to index that peer's catalogue. There is no shared infrastructure and no central registration. Catalogue exchange and trust are outside the initial DAWN charter. This document uses DAWN terminology because cooperating indexes that exchange catalogues need these mechanisms, and is offered as input to the gap analysis, not as a proposal for adoption. 1.1. Scope This document specifies: * a crawl protocol over discovery documents (Section 2); * reciprocal identification of the reading index (Section 3); * an announcement message (Section 4); * an admission model (Section 5); * catalogue preview for peers not yet admitted (Section 5.5). It does not specify payment, identity of providers or operators, reputation or ranking, invocation, or resolution of conflicting listings. 1.2. Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here. 1.3. Terminology Index, catalogue, discovery document, reader: As defined in [DISCOVERY]. Peer: An index known to another index. Artamokhov Expires 3 May 2027 [Page 3] Internet-Draft Index Federation October 2026 Admitted: A peer whose catalogue an index has chosen to index (Section 5). Pending: A peer that an index knows of but has not admitted. Operator: The party that makes admission decisions for an index. 2. Crawl protocol An index discovers peers by retrieving discovery documents, starting from a configured list of URIs (the seed list) and following the peers member of each document retrieved. For each peer it retrieves the discovery document and the catalogue, verifies the catalogue as specified in [DISCOVERY], and stores the result. An index MUST limit the traversal by depth from the seed list and by the number of peers retrieved per cycle, and MUST run the crawl on a schedule rather than in response to queries. An index MUST answer queries from its stored copies of peer catalogues and MUST NOT contact peers while answering a query. Contacting peers at query time would make each query depend on the availability and latency of every peer. An index SHOULD use conditional requests ([RFC9110], Section 13) when it re-retrieves a document, and SHOULD answer them when it serves its own. An index MUST NOT crawl itself. URIs reach the crawl from the seed list, from pending announcements and from the peers of retrieved documents; the self check applies to all three. The comparison MUST normalise scheme, host case and path, because an index may be published under a path of a shared host. 3. Reciprocal identification An index that retrieves another index's discovery document MUST identify itself with the Agent-Marketplace-Crawler request header field (Section 8.1): GET /.well-known/agent-marketplace HTTP/1.1 Host: example.net Agent-Marketplace-Crawler: https://crawler.example The value is the absolute URI [RFC3986] of the retrieving index. This lets an operator learn which indexes read its index. It conveys no trust. Artamokhov Expires 3 May 2027 [Page 4] Internet-Draft Index Federation October 2026 A receiving index: * SHOULD record the value; * MUST treat the value as an unauthenticated claim and MUST NOT grant any trust because of it; * MUST check that the value is an absolute "https" URI before storing it, and MUST apply Section 7.1 before retrieving anything from it; * MAY, under open admission (Section 5.2), record the identified index as a pending peer; * SHOULD limit the number of distinct values it records. Implementations SHOULD process the value outside the request path, so that serving the discovery document does not wait for name resolution. 4. Announcement An index MAY announce itself to another index with an HTTP POST of a JSON object [RFC8259] to an endpoint the receiving index publishes. Members: +====================+==============+=============+ | Member | Type | Requirement | +====================+==============+=============+ | index_url | string (URI) | REQUIRED | +--------------------+--------------+-------------+ | discovery_url | string (URI) | REQUIRED | +--------------------+--------------+-------------+ | capabilities_count | number | OPTIONAL | +--------------------+--------------+-------------+ | signer_public_key | string | OPTIONAL | +--------------------+--------------+-------------+ | signature | object | OPTIONAL | +--------------------+--------------+-------------+ Table 1 If present, signature is a signature object as defined in [DISCOVERY], under the profile named in the announcing index's discovery document, over the canonical string hub_url:|well_known_url:| capabilities_count: Artamokhov Expires 3 May 2027 [Page 5] Internet-Draft Index Federation October 2026 (shown on two lines for width; the string contains no line break) with defaulting to 0. The pair names follow the deployed wire format. The canonical string contains no timestamp or nonce, so an announcement can be replayed. This is acceptable only because an announcement grants nothing (Section 5.3). A verified announcement signature shows only that the sender holds the key it advertises, and MUST NOT be used in an admission decision. In this revision the location of the announcement endpoint is deployment-specific (the reference implementation uses /ai- market/v2/federation/announce). A link from the discovery document may be added in a later revision. 5. Admission An index MUST keep two questions separate: whether it knows that a peer exists, and whether it indexes that peer's catalogue. 5.1. States An index MUST place every known peer in exactly one of these states: Pending: The peer is known. Its catalogue is not indexed, not queryable and not used for routing, and its peers member is not followed. Admitted: The operator has decided to index the peer's catalogue. How the catalogue is then used is outside this document. An index MAY define sub-states of "admitted". It MUST NOT define a state in which a peer's catalogue is used without admission. 5.2. Open and closed admission An index MUST support closed admission, in which a peer becomes pending only by operator action (a seed-list entry, or an announcement the operator has authorised). An index MAY also support open admission, in which an unauthenticated party can create a pending entry by announcement (Section 4) or by reciprocal identification (Section 3). Artamokhov Expires 3 May 2027 [Page 6] Internet-Draft Index Federation October 2026 5.3. Rules With open admission, all of the following MUST hold: 1. An unauthenticated announcement or reciprocal identification creates a pending peer, never an admitted one. 2. An announcement about an already-known peer MUST NOT change any stored property of that peer, including its name, pinned key and state. 3. The number of pending peers MUST be limited. When the limit is reached, further unauthenticated announcements MUST be refused with status 429 (Too Many Requests). 4. An announced or self-identified URI MUST pass Section 7.1 before it is stored. 5. A pending peer MUST NOT be listed in the peers member of the index's own discovery document. Under rule 5, a peer listed in peers is one the operator has admitted. Without it, any party could add a URI to the crawl of every index that reads this one. 5.4. Automatic assessment An index MAY run automated checks on a pending peer (for example, retrieving its discovery document, verifying its catalogue signature, or calling a listed capability) and show the results to the operator. It MAY admit a peer automatically based on such checks. If it does, it MUST record which checks were run and their results, and the operator MUST be able to reverse the admission. A peer that fails the checks remains pending and visible to the operator. 5.5. Catalogue preview An index MAY retrieve and display a pending peer's catalogue so that its operator can decide on admission. If it does: 1. The catalogue MUST be validated, and its signature verified if present, as for an admitted peer. 2. Previewed entries MUST NOT be returned by queries, used for routing, or included in the index's own catalogue. Implementations SHOULD store them separately from the data used to answer queries, rather than rely on filtering. Artamokhov Expires 3 May 2027 [Page 7] Internet-Draft Index Federation October 2026 3. Any interface that shows previewed entries MUST mark them as not admitted. Signature verification of a preview shows only that the catalogue was signed by the key the peer advertises and was not modified in transit. It says nothing about whether the peer should be trusted. 6. Relationship to other work This section is informative. [I-D.mp-agntcy-ads] federates directories through a Kademlia DHT and content-addressed OCI storage, with Sigstore-signed records. That provides global content-addressed lookup, with infrastructure that every participant runs. This document requires no shared infrastructure. A DHT-based directory could publish a discovery document and be admitted like any other peer. [I-D.pioli-agent-discovery] federates through configured trust relationships, which corresponds to closed admission here. This document adds open admission under the rules in Section 5.3. [A2A] defines a signed card per agent and leaves registries to deployments. An index under this document can act as such a registry, and MAY also publish an agent card for itself. [MCPREG] has one upstream registry and sub-registries that mirror it. An index can admit the upstream registry as a peer; this document also covers the case without an upstream. [X402BAZAAR] builds a catalogue from metadata declared in payments that pass through a facilitator. Discovery here does not depend on a prior payment. The pending/admitted distinction is similar to the separation in Certificate Transparency [RFC9162] between an entry being logged and being trusted. 7. Security Considerations 7.1. Request forgery Every mechanism in this document causes an index to retrieve a URI chosen by another party. Before retrieving a URI obtained from a peer's discovery document, an announcement, or the Agent-Marketplace- Crawler field, an index MUST: Artamokhov Expires 3 May 2027 [Page 8] Internet-Draft Index Federation October 2026 * resolve the host and refuse private, loopback, link-local, multicast and unspecified addresses, including IPv4-mapped and IPv4-compatible IPv6 forms, using an explicit list of ranges rather than a library's definition of "private"; * check the resolved address again at connection time, or connect to the checked address; * not follow redirects automatically; * limit response size and time. An index that shares a host with other services MUST NOT treat a URI that resolves to its own host as a peer. 7.2. Key pinning An index MUST verify a peer's catalogue with the key pinned for that peer, as specified in [DISCOVERY]. A changed key is reported to the operator and never accepted automatically. As a result, a key change at a peer requires action by each operator that has pinned it. 7.3. Resource exhaustion Open admission accepts unauthenticated writes; the limits in Section 5.3 bound them. The traversal limits in Section 2 bound the work caused by a large or cyclic peers member. 7.4. Split views An index can serve different catalogues to different readers. A reader that stores the signed catalogue it received can show what it was served, and readers can compare; this document does not define how. A transparency log for catalogues, in the style of [RFC9162], would address this and is out of scope. 7.5. Privacy The record described in Section 3 shows which indexes read a given index. An implementation SHOULD record only the declared URI and SHOULD NOT keep the client network address. 7.6. Limits This document does not cover payment, and a compliant index makes no statement about whether a catalogued capability works as described. Admission is an operator decision, not a guarantee. Artamokhov Expires 3 May 2027 [Page 9] Internet-Draft Index Federation October 2026 8. IANA Considerations 8.1. HTTP field name registration IANA is requested to register the following in the "Hypertext Transfer Protocol (HTTP) Field Name Registry" [RFC9110]: Field name: Agent-Marketplace-Crawler Status: provisional Reference: Section 3 of this document Comments: Request header field. The value is an absolute URI identifying the index on whose behalf the request is made. No X- prefix is used, per [RFC6648]. 9. Implementation Status This section records the status of known implementations at the time of writing, per [RFC7942]. It is to be removed before publication as an RFC. * Reference implementation (Apache-2.0) of the crawl, reciprocal identification, announcement, both admission modes, automatic assessment and preview, with a conformance runner and test vectors. Deployed on six public indexes operated by the author or affiliated parties, with open admission enabled since October 2026. Since 2026-10-30 it sends and accepts Agent-Marketplace- Crawler in addition to the earlier X-AIMarket-Crawler. * A Go directory node implementing crawl, discovery and announcement intake, written by the same author from the text and test vectors without the reference source. * The largest deployed index lists 21 admitted peers. Three services known to it are operated by unrelated parties; as of 2026-10-30, two of them publish catalogues that verify under the profile in [DISCOVERY], and none of them runs a federating index. No implementation of this document by another party is known. 10. References 10.1. Normative References Artamokhov Expires 3 May 2027 [Page 10] Internet-Draft Index Federation October 2026 [DISCOVERY] Artamokhov, A., "A Discovery Document and Signed Catalogue for Capability Marketplaces", Work in Progress, Internet- Draft, draft-artamokhov-dawn-marketplace-discovery-01, October 2026, . [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, . [RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform Resource Identifier (URI): Generic Syntax", STD 66, RFC 3986, DOI 10.17487/RFC3986, January 2005, . [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, . [RFC8259] Bray, T., Ed., "The JavaScript Object Notation (JSON) Data Interchange Format", STD 90, RFC 8259, DOI 10.17487/RFC8259, December 2017, . [RFC9110] Fielding, R., Ed., Nottingham, M., Ed., and J. Reschke, Ed., "HTTP Semantics", STD 97, RFC 9110, DOI 10.17487/RFC9110, June 2022, . 10.2. Informative References [A2A] "Agent2Agent (A2A) Protocol Specification, version 1.0", March 2026, . [I-D.jimenez-dawn-discovery-landscape] Jimenez, J., Feng, J., Arkko, J., Kühlewind, M., and R. Kandoi, "A Survey of AI Agent Discovery Mechanisms", Work in Progress, Internet-Draft, draft-jimenez-dawn-discovery- landscape-00, 3 July 2026, . Artamokhov Expires 3 May 2027 [Page 11] Internet-Draft Index Federation October 2026 [I-D.mp-agntcy-ads] Muscariello, L. and R. Polic, "Agent Directory Service", Work in Progress, Internet-Draft, draft-mp-agntcy-ads-02, 6 July 2026, . [I-D.pioli-agent-discovery] Pioli, R., "Agent Registration and Discovery Protocol (ARDP)", Work in Progress, Internet-Draft, draft-pioli- agent-discovery-01, 24 February 2026, . [MCPREG] "Model Context Protocol Registry", 2026, . [RFC6648] Saint-Andre, P., Crocker, D., and M. Nottingham, "Deprecating the "X-" Prefix and Similar Constructs in Application Protocols", BCP 178, RFC 6648, DOI 10.17487/RFC6648, June 2012, . [RFC7942] Sheffer, Y. and A. Farrel, "Improving Awareness of Running Code: The Implementation Status Section", BCP 205, RFC 7942, DOI 10.17487/RFC7942, July 2016, . [RFC9162] Laurie, B., Messeri, E., and R. Stradling, "Certificate Transparency Version 2.0", RFC 9162, DOI 10.17487/RFC9162, December 2021, . [X402BAZAAR] "x402 Bazaar discovery extension", 2026, . Implementation experience * An index whose seed list contained its own URI recorded itself as a pending peer and contacted itself on every cycle. Its self check had covered only URIs taken from other indexes' peers members; Section 2 now applies the check to all three sources. * A service operated by an unrelated party announced itself, failed the automatic checks because it published no catalogue, and remained visible to the operator as pending and indexed by nothing, as Section 5 intends. It was later admitted. Artamokhov Expires 3 May 2027 [Page 12] Internet-Draft Index Federation October 2026 Changes from -00 Editorial: the text was shortened, and explanatory passages were removed or moved to the appendix above. Implementation Status was updated. No normative requirement changed. Acknowledgements Thanks to the authors of [I-D.jimenez-dawn-discovery-landscape]. Author's Address Aleksandr Artamokhov Independent Email: alexar76@rambler.ru Artamokhov Expires 3 May 2027 [Page 13]