Local & EU AI
For a team whose prompts cannot leave the company, or cannot leave the EU. One senior engineer, scope fixed in a Statement of Work.
Who this is for
A company that wants an internal assistant, or chat over its own documents, or a SaaS team adding a language-model feature — and a US API is not an acceptable place for that data.
The trigger is a customer questionnaire asking where prompts go, a works council that objects, staff pasting documents into ChatGPT, or a token bill that keeps growing.
I map which data reaches which model, pick a setup per use case, and implement the one named in the Statement of Work.
Three ways to run a model
Local or on-prem: the model runs on hardware the company administers. Prompts and documents stay on that machine.
Self-hosted on EU GPUs: the same open-weight model, on a server in the EU the company controls — a Hetzner GPU server, AWS in Frankfurt, or the AWS European Sovereign Cloud.
An EU-hosted model API: a provider runs the model and keeps processing in the EU. Less to operate, and the data does leave the company.
A frontier model stays available only behind a gateway, and only for data that may leave the EU. The gateway is where authentication, routing, and the decision not to log prompts live.
What you get
- ✓Review of which data reaches which model, region, and subprocessor
- ✓A written decision per use case
- ✓A shortlist of models, tested on your real tasks
- ✓An OpenAI-compatible endpoint (vLLM or Ollama) on your hardware, a Hetzner GPU server, AWS Frankfurt, or the AWS European Sovereign Cloud
- ✓A gateway with authentication, routing, rate limits, and no prompt logging
- ✓An internal chat, or chat over your documents, with single sign-on
- ✓A cost comparison: pay per token, or rent a GPU
- ✓Written Statement of Work with fixed price
Local and EU AI questions
What is local AI?
A language model that runs on hardware the company administers: a workstation, a server in the office, or a machine the company controls. Prompts and documents stay on that machine.
Should I run a local model or use an EU API?
Local or on-prem when the data must not leave the company. An EU-hosted API when the data may leave the company but must stay in the EU, and nobody wants to run GPUs. Self-hosting on an EU GPU server sits between the two.
Does an EU data centre make this GDPR-compliant?
No. An EU region is data residency, not a legal verdict. A contract, the subprocessor list, retention, and who can access the system still matter. This engagement is the technical side. It is not legal advice.
What hardware does a local model need?
It depends on the model and the traffic. A small open-weight model can run on one modern GPU. A larger one needs a GPU server with enough memory for the weights and the context. I size it from the tasks, not from a catalog.
Are open-weight models good enough?
For many internal tasks — summarising, extracting, classifying, answering from your documents — current open-weight models are enough. We test that on your real tasks before the Statement of Work commits to one. The hardest reasoning can stay on a frontier model, behind the gateway, for data that may leave the EU.
What does it cost?
The review and the build are priced in the Statement of Work, on request. Running cost is either per token at an EU API, or a rented GPU. The comparison of the two is a deliverable.
First step
The call names the data and the use cases. After that, a bounded Statement of Work.
- Scope
- One product or one internal assistant: the data that may reach a model, and the use cases in scope.
- Deliverables
- A written decision per use case, and the hosting, gateway, or chat we agreed to build.
- Turnaround
- Agreed in the Statement of Work.
- Exclusions
- Training a model from scratch, legal advice, buying hardware, and 24/7 operations.
- Price
- On request.
Tell me where your prompts go
📝 Or send a message