EU Router 03 Serve
Optimized LLM inference for production applications.
GLM
DeepSeek
Qwen
Kimi
Mistral
Llama
gpt-oss
Gemma
Loes
Nemotron
Claude Code
Codex
OpenCode
Cursor
Cline
Loes
OpenAI SDK
Anthropic SDKWhat it does EU Router
Serving a language model to real users is a different job than running one on a laptop. Production inference means concurrent requests, streaming tokens, predictable capacity and a clear view of what every request cost you. HostYourAI runs open models such as Llama, Qwen, Mistral and DeepSeek with vLLM on European GPUs, exposed through an OpenAI-compatible API, so your application code barely changes.
Where that inference happens matters. Every prompt your users send is data, and often personal data. Keeping inference in European datacenters, backed by a Data Processing Agreement and a public subprocessor list, turns a compliance headache into a checkbox you can actually tick.
Read more
01
Support tickets, medical notes, contract clauses: whatever your users type ends up in a prompt. Under the GDPR you are accountable for where that text is processed. Inference that never leaves multiple European regions keeps your data flow map short and your DPA meaningful. HostYourAI does not train on customer data, and traffic is encrypted with TLS in transit and AES-256 at rest.
02
Round trips to another continent add delay to every token of a streamed response. For chat interfaces, where perceived speed is the time to the first token, serving from European GPUs to European users simply feels faster. Popular models on the shared Router are kept warm through a warm pool, so requests do not wait for a cold model to load.
03
Inference bills are notoriously opaque. HostYourAI uses one prepaid credit balance, topped up with iDEAL, card or SEPA, and every request is metered so you can see usage per key. Because the API is OpenAI-compatible, moving in or out is a base URL change, not a rewrite. Details are on the pricing page.
Three things separate a demo from a service. First, streaming: the API supports server-sent events on /v1/chat/completions, so tokens reach your UI as they are generated. Second, capacity: the shared EU Inference Router pools demand across a warm fleet, while dedicated GPU instances give you a vLLM server of your own, with an encrypted API key and a VRAM floor that prevents deploying a model onto a GPU that is too small. Third, observability: per-request usage records and a 99.9% uptime SLA, so you know both what happened and what to expect.
| Production requirement | How it is met |
|---|---|
| Token streaming | SSE on /v1/chat/completions, OpenAI-compatible |
| Fast first token | Warm pool keeps popular Router models loaded |
| Guaranteed capacity | Dedicated vLLM instances billed per minute |
| Usage visibility | Per-request metering from one credit balance |
| Availability | 99.9% uptime SLA |
| Right-sized hardware | VRAM floor enforced at deployment |
Create an API key, pick a model from the model catalog, and point your existing OpenAI client at the new base URL. This JavaScript example streams a response from Llama 3.3 70B:
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://hostyourai.com/api/v1",
apiKey: "hyai-...",
});
const stream = await client.chat.completions.create({
model: "llama-3.3-70b",
messages: [{ role: "user", content: "Summarize this ticket for the support team." }],
stream: true,
});
for await (const chunk of stream) {
process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}The catalog lists over 700 open models, including Llama, Qwen, Mistral, DeepSeek and Gemma. The Model Garden shows the state of each one: warm, warming up, serveable, or on request.
Usually only the base URL and API key. The API is OpenAI-compatible, and there is also an Anthropic-compatible /v1/messages shim for code written against that SDK. See the guide to migrate your OpenAI client to the EU Router.
The Router bills per token and suits variable workloads. A dedicated instance bills per minute, gives you isolated capacity, and stops billing the moment you delete it. Both draw from the same prepaid credit balance.
Popular Router models are kept warm, so there is no load time. Less common models may show as warming up in the Model Garden while vLLM loads the weights.
Yes, /v1/chat/completions supports SSE streaming exactly as the OpenAI API does. Set stream to true and consume the chunks.
Inference runs in European datacenters across multiple European regions. HostYourAI provides a DPA, publishes its subprocessor list, and never trains on customer data.
From model hosting to a customer-facing API, it is built for developers and businesses who want their AI running on infrastructure they actually control, inside the EU.
EU-hosted
100%
Your data and your models stay on European GPUs. GDPR-friendly by design.
Verified models, ready to serve
700+
Llama, Qwen, DeepSeek, Mistral, FLUX and plenty more. Pick one and it is warm in minutes, with no DevOps on your end.
OpenAI & Anthropic compatible
2 SDK
Point your existing client at the Router and keep your tools. No rewrite, no lock-in.
No infra to manage. Pick a model, get an OpenAI-compatible URL, ship.
01
Choose from the Model Garden or paste any HuggingFace ID. Set the VRAM and pick an EU GPU.
02
We deploy vLLM, run readiness probes, and hand you a warm OpenAI- and Anthropic-compatible URL plus an API key.
03
Point your client at the Router. It auto-routes to a warm instance and logs every request.
HostYourAI keeps your models, prompts and data on European GPUs. It is built for teams that care about compliance, reliability and real control.
Full data sovereignty
EU
GPUs and data stay inside Europe, and so do your prompts.
Models you can audit
Open
Run open-weight models and see exactly what runs.
No monthly fee
€0
Prepaid credit per token or per minute, so you pay only for what you run.
Your keys, your models
Yours
Your keys, your models and your data. Move whenever you want.
The Router speaks the OpenAI and Anthropic APIs, so it drops straight into the clients and SDKs your team already runs. Just change the base URL.
Docker
Anthropic
Hugging Face
LangChain
Python
Node.js
curl
Ollama
JetBrains
Jupyter
Vercel
Zapier
Postman
n8nText and image models on dedicated EU GPUs. Every model tested on our own hardware.
Start with one layer and add the others when the work asks for it. They share one key, one balance and one Activity view.
Explore more about EU-hosted AI on HostYourAI.
Model Garden
Browse a curated catalog of serveable open chat, image and embedding models with live warm/EU state. Deploy in one click on European GPUs.
Read moreFine-Tuning LLM
Fine-tune LLMs for your specific use case. LoRA, QLoRA, full fine-tuning on dedicated GPUs.
Read moreOpenAI-compatible API
A drop-in OpenAI-compatible API on European GPUs. Change only the base URL and run open models, GDPR-friendly, without rewriting your code.
Read moreStart with one key today. Add dedicated GPUs when your workload needs them.
