EU Inference Router

What is the EU Inference Router?

In short: we host open AI models on European GPUs and serve them behind one API. Your existing code keeps working, you pay per token, and your data stays in the EU.

The Router is a shared, OpenAI-compatible inference gateway. You point your existing client at a single base URL and the Router sends each request to an open model running on European GPUs. You change only the base URL and the API key; your code stays the same.

Because the Router speaks both the OpenAI and Anthropic APIs, it works directly with the SDKs and tools your team already uses. No rewrite, no vendor lock-in.

How does it work?

A request hits the Router, is authenticated with your API key and forwarded to a model that is already warm. Responses stream back just like OpenAI. Popular models are kept warm through a warm pool so a first request does not wait on a cold start.

Every request records usage, latency and cost in your activity log, so you see exactly what happens.

Why route through it?

• Drop-in OpenAI- and Anthropic-compatible: only the base URL changes
• Open models on European GPUs, managed by us: you run nothing yourself
• Your prompts and data never leave the EU
• Pay-as-you-go per token on one prepaid credit balance
• Per-request insight into usage, latency and cost

Router or dedicated instance?

The Router is the fastest route to production: shared capacity, pay per token and nothing to manage. When you want isolation or predictable performance, you add a dedicated GPU instance next to it. Both work through the same base URL and the same credit balance.

How to start

Create an API key, set the base URL in your existing client and pick a model from the Model Garden. Streaming works the way you expect:

from openai import OpenAI

client = OpenAI(
    base_url="https://hostyourai.com/api/v1",
    api_key="hyai-...",
)

resp = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[{"role": "user", "content": "Hello"}],
    stream=True,
)
for chunk in resp:
    print(chunk.choices[0].delta.content or "", end="")

Need the full step-by-step? Follow the guide Migrate your OpenAI client to the EU Router.

Autopilot: one model id that picks for you

If you would rather not track the catalog every week, put hyai/auto in the model field of your request and the choice is made. Autopilot reads the task instead of the model name you would otherwise have to know: how long the prompt is, what language it is in, whether tools are attached, how much context it needs and what kind of work is being asked. It then looks for the model that fits.

Preference goes to a model that is already warm, because that saves the wait of a cold start. Which model actually answered is in the model field of the response, exactly as with an explicit choice. Want to steer yourself? Fill in a model name and nothing changes.

curl https://hostyourai.com/api/v1/chat/completions \
  -H "Authorization: Bearer hyai-..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "hyai/auto",
    "messages": [{"role": "user", "content": "Summarise this report"}]
  }'

What Autopilot costs

You pay the token rate of the model that answered, with no surcharge for the choice. If Autopilot picks a large model, you pay that large model; if it picks a lighter one, you pay the lighter one. Never more than if you had filled in that same model yourself.

Whether it works out cheaper on average than choosing yourself is something we measure per request: alongside every request we record which model Autopilot would have picked. We publish that number once there are enough requests behind it for it to mean something.

The choosing stays in Europe too

Whoever chooses, reads your prompt. A selector processes your data just as much as the model that answers, and that is exactly where an auto option can quietly leave the EU. With us the choice happens inside the same European chain as the answer itself: your prompt does not leave the EU, not even for the choice.

The choice log keeps only characteristics of the request, such as language, length and whether tools were attached, never the text itself. If you work with a sovereignty key, Autopilot picks exclusively from models that meet that requirement; models that do not qualify never reach the shortlist.

When does your own GPU get cheaper?

The shared Router bills per token and drops back to zero between your requests: send nothing and you pay nothing. A dedicated GPU works the other way around. It bills per minute and stays on around the clock, at night and at the weekend too, but then the machine is entirely yours: no queue behind anyone else's requests, predictable latency and all of the GPU memory for your model.

Where the crossover point sits depends on your monthly volume. In the calculator below you slide that volume along the scale and see, per model class, which of the two is cheaper at that point.

Calculator: when does dedicated become cheaper?

Slide your monthly token volume to where your workload sits and pick a model class. The calculator uses the rates listed above on this page, so the prices we actually bill.

Questions about the EU Router