Build RAG on EU GPUs

What RAG does for you

With Retrieval Augmented Generation (RAG) you ground answers in your own documents instead of only the model's training knowledge. HostYourAI provides the open models and the EU GPU infrastructure; you decide what knowledge goes in. Documents, prompts and answers stay inside Europe end to end.

Step 1, Create a knowledge base

Upload documents (PDF, TXT, CSV, MD or DOCX) or crawl a URL or sitemap. Your content is split into chunks of about 1000 characters with 100 characters of overlap, on sentence boundaries. Each chunk gets an embedding (384-dimensional). Small files (< 50MB) are fully pre-embedded for fast retrieval; large files use a hybrid mode with FULLTEXT plus on-demand embedding.

Step 2, Link the knowledge base

There are three ways to attach a knowledge base to your chat requests:

• Link to an instance (recommended): POST /api/kb/{kbId}/link/{instanceId}. Every chat request to that instance then gets context automatically, including via an external hyai- key.
• Link to an agent: set knowledge_base_id on the agent; the public chat uses it automatically.
• Explicit per request: pass knowledge_base_id in the body of your chat completion.

Step 3, Ask with context

curl https://hostyourai.com/api/v1/chat/completions \
  -H "Authorization: Bearer hyai-..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama-3.3-70b",
    "messages": [{"role":"user","content":"What is our return policy?"}],
    "knowledge_base_id": 42
  }'

The Router retrieves the most relevant chunks with cosine similarity (minimum threshold 0.3), injects the top 5 into the prompt, and generates an answer grounded in your documents.

Source citations

Non-streaming answers include an extra sources array with document name, score, chunk index and an excerpt, so you can trace every answer back to its source. On streaming, context is still injected, but sources cannot be returned in the SSE chunks.

Robust and compliant

Retrieval failures are graceful: if retrieval fails, the chat continues without context instead of erroring. And because everything runs on EU GPUs, your entire RAG pipeline, from documents to answer, stays inside Europe.

Reference: the pipeline settings at a glance

These values determine how your documents are processed and retrieved. They are fixed, so quality stays predictable across all your knowledge bases.

Troubleshooting

My answers do not contain a sources array

Source citations only work on non-streaming answers. Turn stream off for requests where you need the sources; with streaming, context is still injected but the sources do not fit in the SSE chunks.

The model seems to ignore my documents

First check that the knowledge base is actually linked: to the instance, to the agent, or via knowledge_base_id in the request body. Then check that your documents have finished processing; right after an upload, processing is still running in the queue.

I get 401 Unauthorized

Your API key must start with hyai- and be copied in full, and the base URL must be exactly https://hostyourai.com/api/v1. A key from another provider does not work.

The first answer takes long

The chosen model is probably not warm yet. The Model Garden shows whether a model is warm or still warming up; after that, follow-up requests respond quickly.

Questions about this guide