Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Internal / Gossip API

These endpoints carry inter-node CRDT replication traffic and internal service operations. They are served on the same port as the public OAuth2/OIDC endpoints and protected at the application layer — port-level firewall rules cannot isolate them without also blocking public clients.


Gossip endpoints

MethodPathDescription
POST/api/gossip/syncAccept an incoming CRDT gossip push from a peer node. The body is a CMS SignedData(EnvelopedData) blob (ECDSA P-256 outer signature, ML-KEM-768 inner encryption). The receiver verifies the signature, decrypts, applies admission filters, merges the CRDT, and replies with its own state (delta or full, based on the request_delta_since field in the inbound envelope) in the same CMS format.
GET/api/gossip/kem-infoReturn this node’s ML-KEM-768 public key (base64url SPKI DER) and node_id. Unauthenticated. Returns 404 Not Found with {"registered": false} when this node has no CRDT entry yet (i.e. the node has not yet completed its first gossip round); callers should treat 404 as “not yet registered”.
GET/api/gossip/wrapping-keyReturn the cluster AEAD wrapping key sealed to the requester’s ML-KEM-768 public key (SignedData(EnvelopedData) DER, application/pkcs7-mime). The requester identifies itself via the X-Ahdapa-Node-Id header and must have a KEM key already in the CRDT. Unauthenticated at the HTTP level; confidentiality is ensured by the ML-KEM-768 encryption — the blob is useless without the requester’s private key.
POST/api/gossip/register-kemKerberos-authenticated self-registration of both the ML-KEM-768 public key and the ECDSA P-256 gossip signing public key. An IPA-enrolled peer presents Authorization: Negotiate <kerberos-token> (Kerberos AP-REQ for the local HTTP service) and a JSON body {"node_id":"<hostname>","kem_public_key_der":"<base64url-SPKI>","gossip_signing_pub_key_der":"<base64url-SPKI>"}. All three fields are required; missing or empty fields return 400. The server validates that the authenticated principal is HTTP/<hostname>@<REALM>, that node_id matches <hostname>, and (when gossip.kerberos_realm is set) that the principal’s realm matches. Both keys are stored in the CRDT in a three-case match: insert-fresh, upsert-signing-key-only, or no-op (both already present). Returns 503 when the GSSAPI server credential is unavailable or when the DB persist fails; 401 with WWW-Authenticate: Negotiate when no token is presented or invalid; 400 when required fields are missing; 403 on principal or allowlist rejection; and 200 OK on success.
GET/api/gossip/statsReturn runtime gossip statistics for this node. Unauthenticated. Returns a JSON object with node_id, crdt_generation, per-collection live counts under counts, configured and topology-discovered peers, active_signing_kid, kem_enrolled, gossip_signing_enrolled, and a gossip sub-object with started_at, rounds_completed, last_round_at, peer_last_sync (map of peer node_id → last inbound sync unix timestamp), persist_errors, and wrapping_key_pull_errors. See Gossip Protocol — Node statistics endpoint for the full response schema.
GET/api/gossip/await-client?client_id=ID&timeout_ms=MSLong-poll endpoint that blocks until client_id appears in the local CRDT or timeout_ms elapses (default: 30000). Unauthenticated. Uses crdt_changed Notify to detect convergence without polling. Returns 200 OK when the client is found, 408 Request Timeout on timeout. Primarily used by ahdapa-bench converge to measure gossip propagation time.

Credential exchange (OIDC-to-Kerberos)

MethodPathAuthDescription
POST/api/internal/ccacheAuthorization: Bearer <access_token> with krb5:ccache scopeExchange a valid OIDC access token for an exported GSSAPI credential (Kerberos ccache) via S4U2Self constrained delegation.

This endpoint bridges the OAuth2/OIDC world to Kerberos: a FreeIPA server-side component presents a valid access token and receives an exported credential that can be used to perform Kerberos operations on behalf of the authenticated user.

Security model:

  • Localhost restriction: when server.ccache_localhost_only is true (the default), the endpoint rejects requests from non-local source IPs with 403 access_denied. “Local” means loopback addresses or any IP address assigned to a local network interface (enumerated via getifaddrs, cached for 30 seconds). This prevents accidental exposure when the endpoint is reachable over the network.
  • Bearer token: the request must carry a valid JWT access token in the Authorization: Bearer header. The token must contain a sub claim in Kerberos principal form (user@REALM), a client_id claim, an aud claim that includes the client_id, and the krb5:ccache scope.
  • Proof-of-possession (DPoP / mTLS): when the access token contains a cnf claim (RFC 9449 DPoP jkt or RFC 8705 mTLS x5t#S256), the endpoint validates the corresponding proof. DPoP proofs are verified against the endpoint URL {issuer}/api/internal/ccache with method POST. mTLS certificate thumbprints are extracted from the TLS handshake or a trusted proxy header. Missing or mismatched proofs return 401 invalid_token or 401 invalid_dpop_proof.
  • User existence check: before attempting S4U2Self, the endpoint verifies that the sub principal exists in the directory (static users file or FreeIPA LDAP). Unknown users are rejected with 403 access_denied.
  • HBAC enforcement (fail-open): when no HBAC rules have been created, ccache requests pass through (the krb5:ccache scope check alone gates access). Once at least one HBAC rule is created, the full evaluation runs: user/group membership, client match, scope grant (must include krb5:ccache), source network CIDR, MFA requirement (via AMR), and required ACR. In FreeIPA co-deployments, when the built-in allow_all HBAC rule is enabled (the default), HBAC evaluation is skipped entirely for this endpoint – matching FreeIPA’s default open policy. See Identity HBAC Policy – FreeIPA allow_all.
  • Rate limiting: subject to the same per-IP rate limit as other authentication endpoints (server.auth_rate_limit).
  • Audit logging: every ccache issuance and denial is recorded in the audit journal with the event type ccache.issued or ccache.denied respectively, including the denial reason.
  • Credential encryption: the exported credential bytes are encrypted with gssproxy’s master key and are only usable on the same machine. The credential cannot be replayed on a different host.

Request

POST /api/internal/ccache
Authorization: Bearer <access-token>

The request body must be empty (the route enforces a zero-byte body limit).

Response

On success, 200 OK with Content-Type: application/octet-stream. The response body is a complete MIT Kerberos ccache v4 file (version tag 0x0504). The credential is obtained via S4U2Self and stored into a temporary FILE: ccache via gss_store_cred_into; the file bytes are read back and returned as the response body. The consumer can write the bytes to disk and use them with KRB5CCNAME=FILE:/path, or load them into a MEMORY: ccache via krb5_cc_resolve + krb5_cc_initialize + krb5_unmarshal_credentials.

The response includes Cache-Control: no-store, Pragma: no-cache, and Content-Disposition: attachment.

Error codes

StatusErrorCondition
400invalid_requestsub claim missing or not a valid Kerberos principal (user@REALM), or client_id claim missing, or aud does not contain client_id.
401invalid_tokenNo Authorization: Bearer header, JWT is invalid/expired/revoked, or cnf proof-of-possession binding failed (DPoP key mismatch or missing mTLS certificate).
401invalid_dpop_proofA DPoP header was sent but the proof JWT is invalid (bad signature, wrong method/URL, expired).
401use_dpop_nonceDPoP proof JTI was already used (replay).
403access_deniedSource IP is not local (when ccache_localhost_only = true), HBAC denied the request, or the user does not exist in the directory.
403insufficient_scopeToken does not contain the krb5:ccache scope, or HBAC did not grant that scope.
429slow_downRate limit exceeded for the source IP.
500server_errorS4U2Self delegation failed or credential export failed.
503temporarily_unavailableNo GSSAPI credential is available for S4U2Self (server not configured with [gssapi] initiator_principal).

All error responses include a WWW-Authenticate: Bearer error="<error>" header and a JSON body {"error": "<error>"}.

Prerequisites

  1. [gssapi] initiator_principal must be configured so the server can perform S4U2Self constrained delegation.
  2. The HTTP service principal must have ok_to_auth_as_delegate set so the KDC issues forwardable S4U2Self tickets (required for S4U2Proxy):
    ipa service-mod HTTP/<hostname> --ok-to-auth-as-delegate=true
    
    Without this flag the KDC rejects the S4U2Proxy step with EVIDENCE_TKT_NOT_FORWARDABLE.
  3. gssproxy must be running on the same host (when [gssapi] gssproxy = true).
  4. The krb5:ccache scope must be defined via the admin API (PUT /api/admin/scopes/krb5:ccache) and assigned to the calling client.
  5. At least one HBAC rule must grant the krb5:ccache scope to the relevant users/groups for the calling client (or FreeIPA’s allow_all rule must be enabled).

Access control

Gossip endpoints are protected by two independent mechanisms at the application layer. A rogue client that can reach the server port gains nothing from these endpoints without the corresponding cryptographic keys.

CMS authentication and encryption (/api/gossip/sync, /api/gossip/wrapping-key)

Every gossip sync payload is a CMS SignedData(EnvelopedData) structure:

  • The outer ECDSA P-256 signature is verified against the sender’s pinned gossip signing key stored in the CRDT. A node whose key is not pinned receives 401 Unauthorized regardless of the payload contents.
  • The inner ML-KEM-768 encryption is addressed to the receiving node’s public key. The encrypted payload is opaque to any party that does not hold the private key.

For /api/gossip/wrapping-key, the requester’s X-Ahdapa-Node-Id must match a node_id in the admission allowlist (see below); the response is itself an EnvelopedData blob encrypted to the requester’s ML-KEM-768 key, so the cluster wrapping key is never transmitted in the clear.

Node admission allowlist (allowed_node_ids)

The [gossip] section of the configuration controls which node_ids are permitted to exchange CRDT state:

[gossip]
# Static allowlist.  Only node_ids listed here may participate.
allowed_node_ids = ["ipa1.example.com", "ipa2.example.com"]

When ipa_topology = true, the allowlist is extended automatically with all replica hostnames discovered from the IPA replication topology, and updated every ipa_topology_interval_secs seconds. Entries from the static list and the topology-derived list are merged; the union fails closed — a node_id absent from both is denied.

Kerberos authentication (/api/gossip/register-kem)

Self-registration requires a valid Kerberos AP-REQ for the local HTTP service. Only HTTP/<hostname>@<REALM> service principals are accepted; user principals are rejected. When gossip.kerberos_realm is set, cross-realm principals are also rejected. The authenticated hostname is validated against the submitted node_id and against the allowlist before any key material is stored.

See Multi-node Cluster and Gossip Protocol for operational details.