EU Inference Router

What is the EU Inference Router?

The Router is a shared, OpenAI-compatible inference gateway. You point your existing client at a single base URL and the Router sends each request to an open model running on European GPUs. You change only the base URL and the API key; your code stays the same.

Because the Router speaks both the OpenAI and Anthropic APIs, it works directly with the SDKs and tools your team already uses. No rewrite, no vendor lock-in.

How does it work?

A request hits the Router, is authenticated with your API key and forwarded to a model that is already warm. Responses stream back just like OpenAI. Popular models are kept warm through a warm pool so a first request does not wait on a cold start.

Every request records usage, latency and cost in your activity log, so you see exactly what happens.

Why route through it?

• Drop-in OpenAI- and Anthropic-compatible: only the base URL changes
• Open models on European GPUs you control
• Your prompts and data never leave the EU
• Pay-as-you-go per token on one prepaid credit balance
• Per-request insight into usage, latency and cost

Router or dedicated instance?

The Router is the fastest route to production: shared capacity, pay per token and nothing to manage. When you want isolation or predictable performance, you add a dedicated GPU instance next to it. Both work through the same base URL and the same credit balance.

How to start

Create an API key, set the base URL in your existing client and pick a model from the Model Garden. Streaming works the way you expect:

from openai import OpenAI

client = OpenAI(
    base_url="https://hostyourai.com/api/v1",
    api_key="hyai-...",
)

resp = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[{"role": "user", "content": "Hello"}],
    stream=True,
)
for chunk in resp:
    print(chunk.choices[0].delta.content or "", end="")

Need the full step-by-step? Follow the guide Migrate your OpenAI client to the EU Router.

Questions about the EU Router