AI Backends: Cosmo Federation & Apollo Subgraphs on EKS

Table of Contents

Why AI Changed the Shape of the Backend

For a decade, the default backend conversation was about REST endpoints, service boundaries, and relational schemas. AI workloads broke that default in a quiet but structural way.

An AI feature is rarely a single request against a single table. A useful answer usually needs identity, permissions, entity data, semantic search results, aggregated history, and often a model call — assembled into one coherent response with predictable latency and cost.

That is a composition problem, not a database problem. This is exactly why federated GraphQL has become the practical backbone of modern AI backends: it gives agents and clients one typed contract over many domain services, while each team keeps ownership of its own subgraph.

The stack discussed here is deliberately boring in the best sense:

Client / Agent / Copilot surface
        |
        v
WunderGraph Cosmo Router (federated supergraph)
        |
        +--> Subgraph: catalog     (Apollo Server 4 / Express / TypeScript)
        +--> Subgraph: identity
        +--> Subgraph: ai-insights
        |
        v                        (running on AWS EKS)
DynamoDB (ElectroDB)  +  OpenSearch  +  model providers

Every layer has one job. The router composes. The subgraph owns a domain. DynamoDB owns durable entity state. OpenSearch owns retrieval. The model provider owns generation.

1. Cosmo as the Federation Layer

WunderGraph Cosmo is an open-source implementation of GraphQL Federation: a router, a schema registry, composition checks, analytics, and a control plane you can self-host.

The reason it matters for AI backends is that federation solves the “one endpoint, many owners” problem without turning the gateway into a monolith of hand-written resolvers.

What the router actually provides:

• Composition — subgraph schemas are validated and merged into a supergraph. Breaking changes are caught at check time, not at 3 a.m.
• Query planning — the router decides which subgraphs to call, in what order, and merges entity references.
• A single contract — clients, agents, and internal tools talk to one typed graph instead of a mesh of REST calls.
• Observability — traces, field-level usage metrics, and per-operation analytics.
• Governance — persisted operations, rate limits, and schema checks in CI.

The AI-specific advantage is underrated: an LLM agent works dramatically better against a self-describing, typed graph than against a set of loosely documented REST endpoints. The schema is the tool definition. Field descriptions become the prompt context. Persisted operations become the allowlist of safe actions.

Self-hosted vs Cosmo Cloud

For teams already running EKS, the self-hosted router is a normal stateless deployment: a container, a config, an HPA, and an ingress.

Router on EKS

The router is CPU-bound and stateless, which makes it a well-behaved Kubernetes citizen:

• Run 3+ replicas across availability zones with a PodDisruptionBudget.
• Configure the supergraph via the CDN/config poller or a mounted config for air-gapped setups.
• Set explicit resource requests and limits; query planning is CPU-sensitive under burst.
• Terminate TLS at the ALB, keep mTLS or a service mesh for subgraph hops if your compliance model requires it.
• Export traces via OTLP so router spans and subgraph spans live in the same trace.

2. The Subgraph: Apollo Server 4, Express and TypeScript

Each domain is an Apollo Server 4 subgraph. Apollo Server 4 dropped the built-in framework integrations, so the Express wiring is explicit — which is a benefit, because AI workloads need middleware you actually control.

What belongs in the subgraph:

• The domain schema and its federated entity keys
• Resolvers over domain repositories, never raw client SDK calls in the resolver body
• DataLoader instances scoped per request to eliminate N+1 access patterns
• Request context: tenant, user identity, scopes, trace ID, and a per-request budget for model calls

What does not belong in the subgraph:

• Cross-domain joins — that is the router’s job
• Long-running generation — return a job handle, stream elsewhere
• Provider-specific model logic scattered across resolvers

Federated Entities

The @key directive is what makes composition work. A Product defined in the catalog subgraph can be extended by the ai-insights subgraph with a summary or recommendations field, resolved by reference. The client asks one question; the router fans out.

This is the pattern that keeps AI features from contaminating core domains. AI-generated fields live in an AI subgraph with their own SLOs, their own timeouts, and their own failure mode: a null field with an error extension, not a failed page load.

Latency Discipline

An AI subgraph must be explicit about time budgets:

• Set per-resolver timeouts below the router’s client timeout.
• Prefer @defer for expensive generated fields so the shell renders immediately.
• Treat model calls as optional enrichment: the query should still succeed when generation fails.
• Cache aggressively at the field level; most AI enrichment is far less volatile than it looks.

3. DynamoDB with ElectroDB as the Entity Store

DynamoDB is a strong fit for AI backends because the access patterns are known and the traffic is spiky. What it is not good at is being modeled ad hoc.

ElectroDB solves the part teams usually get wrong: it gives TypeScript-typed entity and service definitions over single-table design, generating the key composition, index mapping, and query builders instead of leaving them as string concatenation in application code.

Why this pairing works with a subgraph:

• Entity definitions map cleanly onto GraphQL federated entities.
• ElectroDB Collections let you fetch related entities in one query — the natural backing for a resolver that returns an object plus its children.
• Typed access patterns fail at compile time, which is exactly where key-design mistakes should fail.
• Single-table design keeps p99 predictable under burst, which matters when an agent issues many parallel queries.

Rules that survive production:

1. Design the access patterns from the GraphQL operations, not from an ER diagram.
2. Keep partition keys tenant-scoped for isolation and hot-key avoidance.
3. Use sparse GSIs for filtered listings instead of scanning.
4. Store generated AI artifacts (summaries, embeddings metadata, evaluations) as separate entities with a version and TTL, never as mutations to source records.
5. Use on-demand capacity until traffic shape is genuinely known.

The last point is important for AI features specifically. Generated content must be traceable: which model, which prompt version, which source revision. Overwriting the source record destroys that audit trail.

4. OpenSearch as the Retrieval Layer

DynamoDB answers “give me this entity.” OpenSearch answers “give me the relevant ones.” AI backends need both.

Amazon OpenSearch Service supports lexical search, k-NN vector search, and hybrid search that blends the two. Hybrid is usually the right default: pure vector search misses exact identifiers, part numbers, and error codes, while pure lexical search misses paraphrase.

Indexing pipeline:

DynamoDB table
    |
    v  DynamoDB Streams
Lambda / consumer  --> normalize + chunk --> embed --> OpenSearch index
                                                          |
                                                          v
                                            ai-insights subgraph (hybrid query)

Operational guidance:

• Keep the index eventually consistent by design and expose freshness in the schema rather than pretending it is synchronous.
• Enforce tenant and document-level filters inside the query, not in application code after retrieval. Retrieval that ignores permissions is a data leak with extra steps.
• Version indexes and use aliases so re-embedding is a swap, not an outage.
• Store the chunk’s source key so every generated answer can cite the underlying DynamoDB item.
• Choose between serverless and managed domains based on traffic shape; steady high-volume retrieval is usually cheaper on managed instances.

5. Putting It Together on EKS

A realistic deployment layout:

Platform details that matter:

• IRSA / Pod Identity for DynamoDB and OpenSearch access — no static credentials in the cluster.
• VPC endpoints for DynamoDB so entity traffic never leaves the private network.
• HPA on custom metrics; CPU alone is a poor signal for a service dominated by outbound model latency.
• Separate node groups for AI workloads so a slow provider cannot starve core subgraphs.
• OpenTelemetry end to end — one trace covering router plan, subgraph resolvers, DynamoDB calls, OpenSearch query, and model invocation.

6. Security and Governance

AI backends widen the blast radius of a bad authorization decision, because a single query can compose data from several domains and then feed it into generation.

• Authorize in the subgraph, not the router. The router composes; it should not be the trust boundary for domain rules.
• Propagate identity, never re-derive it. Pass verified claims through as context; subgraphs must not mint their own trust.
• Persisted operations in production. Free-form queries from clients and agents are an availability and exfiltration risk.
• Depth, complexity, and rate limits at the router edge.
• Filter retrieval by permission at query time, always.
• Log model invocations with prompt version, model ID, tenant, and cost attribution.

7. Migration Path

Phase 1: Establish the Graph

• Stand up the Cosmo router and one Apollo Server 4 subgraph on EKS
• Wire schema checks and composition into CI
• Add OpenTelemetry tracing across router and subgraph

Phase 2: Model the Data Properly

• Define ElectroDB entities from real GraphQL operations
• Move resolvers onto typed repositories with per-request DataLoaders
• Add tenant-scoped keys and sparse GSIs

Phase 3: Add Retrieval

• Build the DynamoDB Streams indexing pipeline into OpenSearch
• Start with hybrid search and permission filters at query time
• Version indexes behind aliases

Phase 4: Add AI Fields Safely

• Create a dedicated AI subgraph extending existing entities
• Use @defer, timeouts, and null-tolerant AI fields
• Persist generated artifacts with model and prompt versions
• Track cost per successful operation, not cost per token

Summary

The new era of AI backends is not defined by which model you call. It is defined by how cleanly you compose data, retrieval, and generation behind one typed contract.

Cosmo provides the federation layer and governance. Apollo Server 4 subgraphs on EKS keep domain ownership intact. DynamoDB with ElectroDB gives predictable, typed entity access. OpenSearch supplies hybrid retrieval with permission-aware filtering.

The architecture works because each layer stays replaceable. Model providers will change. Retrieval strategies will change. A federated graph over well-modeled entities is the part that survives.