AI without GPU is possible — with the right CPUs
Not everyone needs an H100. Quantized LLM inference (GGUF in Q4/Q5), light fine-tuning of small models, scikit-learn pipelines, embedding vectorization: all of this runs remarkably well on CPU, provided you have many fast cores and enormous memory bandwidth. That's precisely the profile of our EPYC chips with 12 DDR5 channels: 460 GB/s theoretical per socket.
LLM inference on Singularity
A Llama 3.1 8B in Q5_K_M takes ~6 GB and runs at 15-20 tokens/second on 16 EPYC vCores with llama.cpp — plenty for a business chatbot or internal agent. A Mistral 7B in Q4 exceeds 25 tok/s. On Singularity (32 vCore, 64 GB), you can serve a quantized 13B or several 7-8B models in parallel behind Ollama or CPU vLLM.
DDR5 makes a measurable difference
LLM inference is memory-bandwidth-bound, not compute-bound. Our internal measurements: +38% tokens/second on our DDR5-4800 platform versus a DDR4-3200 platform with identical model and cores. That's the hardware reason two "8 vCore" VPS are not equal.
Real use cases among our customers
Support ticket classification (DistilBERT fine-tuning), RAG answer generation with pgvector embeddings, lead scoring, automatic summarization of legal documents. The common thread: batch or low-concurrency workloads, where a GPU would be wasted.
When to move to GPU
If you serve high-concurrency real-time (100+ simultaneous requests) or fine-tune models > 13B, you'll need a GPU. For everything else — and that's 90% of business use cases — a Singularity CPU is four times cheaper for equivalent service.