3× NVIDIA Tesla V100 (32 GB HBM2 each, 96 GB total) passed through to your VM, with 24 dedicated cores, 105 GB RAM and 1200 GB NVMe: 3× the single-card plan that Strata was measured on, so every card has the RAM and SSD it needs. Run 3 independent Strata servers, one per card, each at the single-card speed (72–79 tok/s), or pool the cards for one bigger job. Driver preinstalled, the cards are yours alone.
| GPU | 3 × NVIDIA Tesla V100 32 GB HBM2 · 96 GB VRAM total — full PCI passthrough, all 3 cards in your VM |
|---|---|
| CPU | AMD EPYC Gen 2 |
| Dedicated Cores | 24 |
| RAM | 105 GB |
| Storage | 1.2 TB NVMe |
| Platform | KVM |
| Operating Systems | Linux, Windows |
| Bandwidth | Option 1: Unlimited transfer — 250 Mbps port, grows +1 Mbps/day up to 1 Gbps Option 2: 10 Gbps port — 100TB 10Gbps monthly cap (throttled to ~10 Mbps after cap) |
Real numbers measured on our own VMs on three NVIDIA Tesla V100s, not estimated: Strata first (the 125B model this plan is built for), then llama.cpp with smaller models. GPU plans run on Proxmox with full PCI passthrough: the card is yours, not shared and not virtualized.
Specced for Strata · measured on our own VMs, 17–19 Oct 2026
Qwen3.8-Flash-Next · 125B mixture-of-experts frontier model (6B active per word) · ISTA-DASLab GSQ-RCO IQ3_S · 128K context · Strata v0.1.44.1
Strata on 3 cards pooled into one server was not measured, so we do not quote a pooled number. What this plan is specced for is 3 separate Strata servers: every card has its own 35 GB of RAM and 400 GB of NVMe, the shape the single-card figures were measured on.
| Prompt | 3.8K tokens | 35K tokens | 87K tokens | 162K tokens |
|---|---|---|---|---|
| Reading | 1,163 tok/s | 1,301 tok/s | 1,404 tok/s · at 40 GB RAM | 1,228 tok/s · 256K context, 32 GB RAM |
| Writing | 72 tok/s | 75 tok/s | 69 tok/s | 62 tok/s |
How we run Strata on these cards, step by step →
Strata v0.1.44.1 (commit 296b59d2), ISTA-DASLab GSQ-RCO IQ3_S, CUDA 12.8.93, driver 580.178.04, Ubuntu 26.04.1, AMD EPYC 7532 vCPUs. Prompt reading and decode are the engine's own timings, mean of 2 runs per size, 512 tokens out, thinking off.
A 125B frontier mixture-of-experts model, entirely in card memory.
Team figures are Qwen3.6-35B-A3B on one card. Three-card figures are from production work on our own servers.
With Strata, Qwen3.8-Flash-Next (125B) is the headline. With llama.cpp, the extra room is what makes this card: higher-precision builds, long context windows and large dense models fit on one card at full speed. Memory is what the model actually used on one card.
| Model and setup | Card memory used | Context | Speed |
|---|---|---|---|
| Specced for · Strata Qwen3.8-Flash-Next (125B) Strata · ISTA-DASLab GSQ-RCO IQ3_S · 128K context · the whole card plus 35 GB of system RAM holding the model's experts. | whole card + 35 GB RAM | 128K | 72–79 tok/s writing |
| Our pick for documents Qwen3.8-35B-A3B Distill Newest generation · mixture of experts · Q4_K_M · a full 128K context (~200 pages). Reads about 2,300 tokens a second. | 27.2 GB | 128K | up to 154 tok/s writing |
| Higher precision Qwen3.8-35B-A3B Distill, 5-bit Q5_K_M · the same model at higher precision, still with a 128K context. | 30.6 GB | 128K | up to 150 tok/s writing |
| Longest context Qwen3.6-35B-A3B Mixture of experts · UD-Q4_K_M · the model's full 262K window (~400 pages) at full speed. | 30.3 GB | 262K | up to 132 tok/s writing |
| Dense, 8-bit Qwen3.8-27B Dense · Q8 · the highest practical precision for a 27B model, with its built-in accelerator. | 31.5 GB | 32K | up to 47 tok/s writing |
| Dense, 4-bit Qwen3.8-27B Dense · 4-bit with its built-in accelerator · reads a 10-page document in about 10 seconds. | 20.2 GB | 32K | up to 59 tok/s writing |
| Dense, 4-bit Muse Glimmer 30B Dense · 4-bit with its DFlash draft model. | 19.5 GB | 32K | up to 60 tok/s writing |
Mixture-of-experts models use only about 3B of their 35B parameters per word, which is why they write so fast. Dense models use every parameter, so they write slower; the V100's tensor cores still read their prompts at 750–1,000 tokens a second.
Serving a team? Add -np 8: with Qwen3.6-35B-A3B, eight people typing at once each get about 33 tok/s, and the card's total output reaches about 210 tok/s. Four 10-page documents, all answered in 38 seconds.
Teach a model your own style, terminology or task. With 32 GB, the V100 has room to fine-tune larger models than a 24 GB card can: QLoRA training of a 27B model such as Qwen3.8-27B needs about 26–28 GB, and fits on one V100. Unsloth Studio, the desktop training app, makes it point-and-click: pick a model, load your dataset, press train. Your data and the trained model never leave your VM.
We are measuring training on the V100 now and will publish the figures here. The V100 trains in FP16 (it has no BF16), which Unsloth supports.
With 32 GB, the V100 keeps a 128K or even 262K context on the card, so the model reads the whole document at once instead of in pieces, and answers about all of it. We measured it with a real 82,000-token document, about 120 pages.
Reading speed on mixture-of-experts models comes from our V100-tuned build of llama.cpp (see Setup). The stock build reads about half as fast.
On Ubuntu the driver is already there. Add Docker and the NVIDIA Container Toolkit, download the model once (21.7 GB), then start the server.
mkdir -p /opt/models && curl -L -o /opt/models/Qwen3.8-35B-A3B-Distill-Q4_K_M.gguf \
https://huggingface.co/empero-ai/Qwen3.8-35B-A3B-Distill-GGUF/resolve/main/Qwen3.8-35B-A3B-Q4_K_M.gguf
docker run -d --name llm --gpus all -v /opt/models:/models -p 127.0.0.1:8080:8080 \
ghcr.io/ggml-org/llama.cpp:server-cuda-b11580 \
-m /models/Qwen3.8-35B-A3B-Distill-Q4_K_M.gguf \
--host 0.0.0.0 --port 8080 \
-c 131072 -np 1 -t 6 -tb 6 -b 4096 -ub 4096 \
-ngl auto --fit on --fit-target 1024 \
--spec-type draft-mtp --spec-draft-n-max 2 --jinja-ub 4096--spec-draft-n-max 2GGML_CUDA_FORCE_MMQThe server speaks the OpenAI API on 127.0.0.1:8080/v1: LangChain, LlamaIndex, Open WebUI and Continue work unchanged. Keep it behind your own proxy with authentication.
Figures: llama.cpp b11580 on Ubuntu 26.04, one Tesla V100-PCIE-32GB per VM, 14–15 Oct 2026 · Qwen3.8-Flash-Next on 3 × V100, real workload, Oct 2026 · on the same models and prompts the V100 measured about 2.5× faster than a Tesla P40 on average · full benchmark report on request.
Two options per VPS, chosen at checkout: (1) Unlimited transfer starting at a 250 Mbps port that grows +1 Mbps/day up to 1 Gbps; or (2) a 10 Gbps port with a per-plan monthly transfer cap (throttled to ~10 Mbps after the cap).
Run Linux or Windows on any VPS — selected at deployment.