Security & Deployment
Hosted API, private cloud or on-premises: deciding where your models run
A comparison of deployment options for language models, covering data protection, quality, cost, latency and the operational burden of each.
2 min readLast reviewed
One of the first questions in any enterprise AI project is where the model runs and where the data goes. The answer affects quality, cost, latency, compliance and how much your team has to operate. There is no universally right choice; there is a right choice for each workload.
The main options
| Option | Strengths | Trade-offs |
|---|---|---|
| Commercial model API | Strongest models, no infrastructure, fast to start | Data leaves your environment under provider terms; dependency on provider roadmap |
| Cloud-hosted model service in your account or region | Enterprise terms, regional processing, private networking | Model choice limited to what the platform offers; cost model varies |
| Self-hosted open-weight models in your cloud | Full control of data and versions | GPU cost and capacity planning; you operate it; quality may lag frontier models for some tasks |
| On-premises | Maximum isolation | Highest operational burden; hardware lead times |
Questions that decide it
- What data will the model see, and what do your contracts, regulators and data protection obligations say about where it may be processed?
- Do the provider’s terms exclude training on your data, and for how long are inputs retained?
- What quality does the task need? Measure candidate models on your evaluation set rather than relying on public leaderboards.
- What latency and volume do you expect, and what does that imply for cost under each option?
- Who will operate, patch and monitor a self-hosted deployment?
Mix options per workload
Many organisations use more than one option: a frontier API for low-sensitivity drafting, a regional cloud service for customer data, and a small self-hosted model for high-volume classification. A routing layer and a common evaluation suite make this manageable.
Data protection is about the whole system
Where the model runs is only part of the picture. Logs, vector indexes, caches and evaluation datasets also contain personal or confidential data. Regulators such as the UK Information Commissioner’s Office publish guidance on applying data protection principles to AI systems, including minimisation and security, which applies across these components.