Deploy your own LLM with vLLM

Router or dedicated instance?

The shared Router is perfect to start quickly and pay per token. Choose a dedicated instance when you want isolation, predictable latency or a specific model on your own hardware. On a dedicated instance, vLLM runs on a single European GPU that is exclusively yours.

Step 1, Pick your model

Select a model from the Model Garden or paste a HuggingFace ID, for example Qwen/Qwen3-8B. Every model has a VRAM footprint; the deploy step estimates the minimum VRAM and stops you from picking a GPU that is too small. We request gated models (such as some Llama variants) centrally, so you do not have to arrange individual licences.

Step 2, Configure the GPU

• Choose the VRAM and a European GPU region
• We source the capacity from European GPU marketplaces
• We generate a vLLM setup script automatically and start the deploy

Step 3, What happens behind the scenes

The GPU fetches the setup script through a signed token and starts vLLM with its own API key (stored encrypted), so your endpoint is secured. We run readiness probes on /health; as soon as vLLM is ready, the instance calls back and the status flips to "warm". You get an OpenAI- and Anthropic-compatible URL plus an API key.

Step 4, Send your first request

Reach the instance through the Router by its model slug, or directly on the instance endpoint:

curl https://hostyourai.com/api/instances/<instance-id>/chat/completions \
  -H "Authorization: Bearer hyai-..." \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"Hello"}]}'

Because the instance is OpenAI-compatible, the OpenAI SDK works here too: set the base URL to the Router and pick your model slug.

Cost and management

A dedicated instance bills per hour, but idles when nobody is online: you do not pay for unused time and the instance warms back up on the first request. Your dashboard shows the status, usage and logs. To add context from your own documents, link a knowledge base to the instance, see the RAG guide.

VRAM guidelines per model size

How much VRAM you need depends on the parameter count, quantisation and context length. The table below gives practical guidelines; the deploy step checks the floor automatically, so you can never pick a GPU that is too small. The current model catalogue is in the Model Garden.

Troubleshooting

My deploy stays in "warming" for a long time

Large models first have to download and load tens of gigabytes of weights, which takes a while. The instance reports back on its own once the readiness check on /health passes. If the status is stuck for hours, check the logs in your dashboard.

The deploy rejects my GPU choice

Then the VRAM is too small for the chosen model. We estimate the minimum VRAM up front and block undersized GPUs, because vLLM would otherwise crash while loading. Pick a GPU with more memory or a smaller model.

I get 401 on my instance endpoint

Every instance runs vLLM with its own API key, stored encrypted. If you go through the Router, use your hyai- key; it must be complete and without spaces in the Authorization header.

My requests return 404

Check the endpoint: through the Router the base URL is https://hostyourai.com/api/v1 with your model slug, and directly you use the instance path /api/instances/<instance-id>/chat/completions.

Questions about this guide