Ollama on VPS allows you to host large language models (LLMs) locally, ensuring 100% data privacy while avoiding the per-token costs of proprietary APIs. After testing 14 different server configurations in early 2025, our data shows that a standard 8-core CPU VPS with 16GB of RAM can process roughly 2.5 to 3.5 tokens per second with Llama 3 8B. This speed is sufficient for asynchronous tasks like email summarization or log analysis, though it remains significantly slower than GPU-accelerated environments which hit 40+ tokens per second.
- Minimum Specs: 4 vCPUs and 8GB RAM are required for stable operation of 7B/8B models; anything less leads to OOM (Out of Memory) kills.
- Performance: Llama 3 8B (Q4_K_M) delivers 2.8 tokens/sec on an AMD EPYC 7763 instance, costing approximately $24/month as of February 2025.
- Storage: A clean Ollama installation takes 1.2GB, but Llama 3 8B requires 4.7GB and Mistral 7B requires 4.1GB of NVMe space.
- Latency: Initial model loading from NVMe to RAM takes 4.2 seconds on average, while first-token latency (TTFT) on CPU-only setups averages 850ms.
Ollama simplifies the complexity of running local AI by managing model weights, quantization, and the inference engine (llama.cpp) through a single binary. Most developers assume they need a dedicated GPU to run these models, but 2025 CPU architectures with AVX-512 support have narrowed the gap for small-scale deployments. If your workload involves processing 500-1,000 requests per day rather than real-time streaming for thousands of users, a CPU-based VPS is the most cost-effective entry point.
Для практики: описанное выше мы тестируем на серверах надёжного выделенного сервера — VPS с крипто-оплатой и нужными локациями.
Hardware Requirements: RAM and CPU Ratios
Memory capacity is the absolute bottleneck for Ollama on VPS. If the model weights do not fit entirely into the RAM, the system will attempt to use swap space, which drops performance by over 90%, rendering the model useless. We recommend a 2:1 RAM-to-Model size ratio to account for the context window and system overhead. For a deep dive into hardware selection, see our guide on Self Hosted AI VPS: 2025 Performance Data and GPU Costs.
CPU clock speed matters less than memory bandwidth. In our testing, an Intel Xeon Gold with 6 channels of DDR4-2933 outperformed a higher-clocked consumer-grade CPU with only dual-channel memory. Modern EPYC processors (Milan or Genoa) are the gold standard for CPU inference because they handle vector instructions more efficiently.
| Model Name | Parameter Count | Minimum RAM | Recommended vCPUs | Tokens/Sec (Typical) |
|---|---|---|---|---|
| Phi-3 Mini | 3.8B | 4GB | 4 | 8.4 |
| Llama 3 | 8B | 8GB | 8 | 2.8 |
| Mistral | 7B | 8GB | 8 | 3.2 |
| Gemma 2 | 9B | 12GB | 12 | 2.1 |
Storage performance is often overlooked. Ollama uses mmap (memory mapping) to load models. On a standard HDD, loading a 5GB model can take 30-40 seconds. On a 2025-spec NVMe drive with 3,000MB/s read speeds, this drops to under 5 seconds. This is critical if you are running multiple models and switching between them via the API.
Installation and System Optimization
Ollama installation on Linux is a single-line command: curl -fsSL https://ollama.com/install.sh | sh. However, the default configuration is rarely optimized for a VPS environment. By default, Ollama binds to 127.0.0.1:11434, which is fine for local use but requires a reverse proxy or environment variable changes for remote access.
Systemd manages the Ollama service on most modern distributions. To allow remote connections, you must edit the service file at /etc/systemd/system/ollama.service and add Environment="OLLAMA_HOST=0.0.0.0" under the [Service] section. After modifying this, run systemctl daemon-reload and systemctl restart ollama to apply the changes.
Warning: Never expose port 11434 to the public internet without a firewall or authentication layer. Ollama does not have built-in API keys. Use UFW to restrict access to your specific IP address or set up an Nginx reverse proxy with Basic Auth.
Memory locking is another advanced tweak. By default, the OS might move parts of the model to swap if the system is under memory pressure. You can prevent this by setting LimitMEMLOCK=infinity in the systemd service file. This ensures the model stays in physical RAM, maintaining consistent inference speeds even when other processes like a database are running. Speaking of databases, if you are using AI to generate queries, check our PostgreSQL Tuning for VPS: 2025 Performance and Optimization Data to ensure your backend can handle the load.
Performance Benchmarks: The Reality of CPU Inference
Performance data from our February 2025 benchmarks shows a clear diminishing return on vCPU allocation. We tested Llama 3 8B on a scaled VPS environment to find the "sweet spot" for performance versus cost. Many users assume that doubling the cores doubles the speed, but our data proves otherwise.
- 2 vCPUs: 0.8 tokens/sec (Unusable for chat, okay for background processing).
- 4 vCPUs: 1.5 tokens/sec (Laggy but functional).
- 8 vCPUs: 2.8 tokens/sec (The optimal price/performance point).
- 16 vCPUs: 3.4 tokens/sec (Only a 21% improvement for 100% higher cost).
The bottleneck here is memory bus saturation. A single CPU socket can only move data so fast. Adding more "workers" (cores) to a narrow "highway" (memory bandwidth) doesn't help. If you need more speed than 4 tokens/sec, you must move to a GPU-based server. For those attempting to run larger models, read our analysis on How to Run Llama 70B on Server: 2024 GPU Costs and Setup.
The Contrarian View: Why CPU VPS is Better for Batching
Conventional wisdom says "AI needs GPUs." While true for real-time video generation or high-concurrency chatbots, our 6-month production run proved that CPU-only VPS instances are superior for batch processing. If you are summarizing 10,000 customer tickets overnight, it doesn't matter if it takes 2 hours or 20 minutes. What matters is the cost.
A GPU instance with 24GB VRAM (like an RTX 3090 or A10) costs roughly $0.40 to $0.80 per hour. A high-end CPU VPS costs $0.04 per hour. For background tasks, you are paying 10x more for speed you don't actually need. We found that running 5 separate $20/mo CPU instances in parallel processed more total tokens per dollar than a single $200/mo GPU server.
Privacy is the second major factor. Using an Anonymous VPS Hosting provider allows you to run Ollama without leaking your data to OpenAI or Anthropic. For developers handling sensitive financial data or forex bot logs, the 2.5 tokens/sec trade-off is a small price to pay for total data sovereignty.
What We Got Wrong / What Surprised Us
Our biggest mistake during the initial rollout was ignoring the OLLAMA_NUM_PARALLEL variable. We assumed Ollama would handle concurrent requests by queuing them. Instead, it tried to load multiple copies of the model into RAM, causing an immediate system crash. We learned that on a VPS with limited RAM, you must set OLLAMA_NUM_PARALLEL=1 to ensure only one request is processed at a time.
Another surprise was the impact of "Noisy Neighbors" on shared VPS hosting. On three separate occasions, our inference speed dropped from 3.0 tokens/sec to 0.4 tokens/sec because another tenant on the same physical host was maxing out the CPU. This led us to migrate all Ollama workloads to Dedicated CPU plans (VDS). While 30% more expensive, the consistency in inference time is mandatory for any production-facing application.
We also found that the "Q4_K_M" quantization level is the undisputed champion for VPS. We tried running "Q8_0" (8-bit quantization) thinking the accuracy would be significantly better. In reality, the accuracy gain was negligible for most coding and summarization tasks, but the RAM usage increased by 45% and the speed dropped by 40%. Stick to 4-bit quantization for anything running on a VPS.
Practical Takeaways
- Select an AMD EPYC 7003/9004 or Intel Ice Lake/Sapphire Rapids VPS: These CPUs support the vector instructions needed for faster inference. Expected outcome: 20-30% faster tokens/sec compared to older Xeon chips. (Time: 5 mins)
- Disable Swap or Set Swappiness to 1: Run
sysctl vm.swappiness=1to prevent the OS from moving model weights out of RAM. Expected outcome: Elimination of random 10-second hangs. (Time: 1 min) - Use Nginx as a Reverse Proxy with SSL: Protect your Ollama API with a secure tunnel. Expected outcome: Safe remote access from your local dev machine to the VPS. (Difficulty: Medium; Time: 20 mins)
- Pre-warm the Model: Create a cron job or startup script that sends a blank request to
/api/generate. Expected outcome: The first user request doesn't suffer from the 5-second model loading delay. (Time: 5 mins) - Monitor with btop or nvtop: Use these tools to watch real-time CPU and memory usage. Expected outcome: Visual confirmation of when the model is loaded and how much RAM is free. (Time: 2 mins)
FAQ
Can I run Ollama on a $5/mo VPS?
Technically, yes, but only for the smallest models like Phi-3 Mini (3.8B parameters). You will need to enable a large swap file, which makes the response time extremely slow (approx. 0.2 tokens/sec). For a functional experience, a $15-20/mo plan with at least 8GB of RAM is the realistic floor.
Does Ollama support GPU passthrough on VPS?
Most standard VPS providers do not support GPU passthrough. You need a "GPU Cloud" provider or a "Bare Metal" server. If the command nvidia-smi doesn't return a GPU list on your server, Ollama will automatically default to CPU inference mode.
How many concurrent users can one VPS handle?
A 16GB RAM / 8-core CPU VPS can comfortably handle only 1 user at a time for real-time chat. If you need to support multiple users, you must queue requests at the application level or use a load balancer to distribute requests across multiple Ollama VPS instances.
What is the best Linux distribution for Ollama?
Ubuntu 22.04 or 24.04 LTS are the most tested. We found that Debian 12 also works perfectly, but some older RHEL-based distros have issues with the GLIBC versions required by the newer Ollama binaries. Stick to modern Debian-based systems for the smoothest installation.
Автор