The constraint

Two enterprise customers. Same product. Different cloud preferences:

• Customer A: AWS only. Uses Bedrock for LLM, S3 for storage, EKS for compute.
• Customer B: GCP only. Uses Vertex AI for LLM, GCS for storage, GKE for compute.

You can’t fork the codebase. You can’t reasonably maintain two separate deployments. The provider router pattern solves this.

The shape

type LLMProvider interface {
    Generate(ctx context.Context, prompt string) (Response, error)
    Embed(ctx context.Context, text string) ([]float32, error)
    Stream(ctx context.Context, prompt string) (<-chan Token, error)
}

type BedrockProvider struct { client *bedrock.Client }
type VertexProvider struct { client *aiplatform.Client }
type OllamaProvider struct { url string }

// Genie picks at boot:
var llm LLMProvider
switch os.Getenv("GENIE_LLM") {
case "bedrock":
    llm = bedrock.New(awsCfg)
case "vertex":
    llm = vertex.New(gcpCfg)
case "ollama":
    llm = ollama.New(os.Getenv("OLLAMA_URL"))
}

The agents never touch the provider directly. They take the LLMProvider interface. Swapping deployment target is one env var.

What’s portable vs not

Portable across providers: - Generate, embed, stream calls. - Token counting (roughly — providers’ tokenisation differs by ~10-15%). - The prompt itself (a good prompt works across models).

Not portable: - Model-specific features (Bedrock guardrails, Vertex safety settings). Wrap them per-provider, expose via a unified safety layer. - Function calling shape (the JSON differs subtly). Normalise at the provider boundary. - Pricing model (per-token vs per-request vs subscribed). The cost wrapper has per-provider logic.

Storage abstraction

Same shape for blob storage:

type Blob interface {
    Put(ctx context.Context, key string, data []byte) error
    Get(ctx context.Context, key string) ([]byte, error)
}

// S3, GCS, Azure Blob each implement the interface.
// The Gocloud library does this for you; we wrap it lightly for typed errors.

What about compute?

You don’t abstract Kubernetes. EKS and GKE differ in cluster setup (auth, networking, autoscaling), but pods running the same container are pods. The container is portable; the cluster setup is per-cloud Terraform.

That’s the right split: portable application; per-cloud infrastructure.

What changes per customer

• The env vars (LLM provider, storage provider, KMS).
• The Terraform / Helm overlay (per-cloud auth, per-cloud networking).
• The data residency policy (per-cloud regions in sovereignty registry).

That’s it. The application binary is the same.

What I learned

The provider-router pattern is unglamorous; it’s also the only way I’ve gotten multi-cloud-from-day-one to work without forking. Single codebase; per-customer deployment; predictable maintenance cost.

For Genie’s roadmap, the AWS variant lives behind GENIE_LLM=bedrock GENIE_STORAGE=s3. The Vertex variant lives behind GENIE_LLM=vertex GENIE_STORAGE=gcs. Same git tag; different env.