Use case · measured on our own VMs, not estimated

AI training & inference on a GPU that is yours

Rent a VPS with a whole NVIDIA card passed through to it, specced so that Strata runs Qwen3.8-Flash-Next (125B) on that one card at full speed. Run open models, serve your team, read your documents and fine-tune on your own data. Nothing leaves the VM, and there is no per-token bill.

Strata: Qwen3.8-Flash-Next (125B) on one card Set it up: Strata or llama.cpp

Why ServaRica fits

You own your data. This is a server product: nobody sees your prompts, your documents or what you run on it. Work on private data without handing it to an AI company that can train on it or build a profile of you.
Full control. Switch to any model you want, whenever your needs change, and build real skills in running AI instead of renting someone else's API.
Canadian data stays in Canada. If you are Canadian, this is one of the rare setups where your data never leaves the country: Montreal, on hardware we own.
GPU plans run on Proxmox with full PCI passthrough: the card is yours, not shared and not virtualized.
Flat monthly price. No per-token, per-request or per-GPU-hour bill, and no rate limit but the card's own speed.
Every number on this page was measured on our own VMs with llama.cpp and Unsloth Studio.
Linux or Windows with the driver preinstalled. Dedicated cores, NVMe, and 100 TB a month on the 10 Gbps port or unlimited transfer on the growing port.

Specced for Strata

Qwen3.8-Flash-Next (125B) on one card, at full speed

These specs are not random. We ran Strata on both cards, found the RAM where Qwen3.8-Flash-Next (125B) runs at full speed (40 GB on the P40, 35 GB on the V100), the SSD it needs (78 GB of model files, a 56 GB prepared copy, swap and the OS) and the cores it actually uses, then built the plans around exactly that. The specs are made to maximize the performance-to-price ratio.

Specced for Strata · measured on our own VMs, 17–19 Oct 2026

Qwen3.8-Flash-Next (125B) at 39–41 tokens a second, on one P40.

Qwen3.8-Flash-Next · 125B mixture-of-experts frontier model (6B active per word) · ISTA-DASLab GSQ-RCO IQ3_S · 128K context · Strata v0.1.44.1

39–41tok/swriting · Qwen3.8-Flash-Next · 128K context

~450tok/sreading a prompt

125BQwen3.8-Flash-Next, a frontier mixture-of-experts model, on one card

Qwen3.8-Flash-Next (125B), a frontier mixture-of-experts model, with a 128K context on one 24 GB card
Decode stays flat from a short prompt to 87K tokens (40 → 39 tok/s)
Needs 40 GB of system RAM for full speed; this plan carries 40 GB (48 GB gave the same speed)
Pictures through the CPU encoder were not measured on the P40; plan about 2 GB more RAM

What that means in practice (worked out from the measured speeds above):

For example, it can read a 50-page document (35K tokens) in about 77 seconds
For example, it can read a 130-page document (87K tokens) in about 3 minutes 20 seconds
For example, at 40 tok/s it writes about 30 words a second, faster than most people read

Measured speed by prompt size, 1 × P40: prompt reading / writing in tokens per second
Prompt	3.8K tokens	35K tokens	87K tokens
Reading	446 tok/s	454 tok/s	430 tok/s
Writing	40 tok/s	41 tok/s	39 tok/s

For comparison, llama.cpp on the same model file and two P40s managed 186 / 26 tok/s at 3.8K and 118 / 17 at 87K. Strata on one P40 reads 2.4× faster and writes 1.5× faster; at 87K it reads 3.6× faster and writes 2.3× faster.

What Strata needs on the VM, and what we tested and do not recommend

Ubuntu 26.04 (tested), NVIDIA driver 580, CUDA 12.8 toolkit (CUDA 13 dropped the P40; 12.9 does not build on this glibc), gcc 14
An SSD marked as such in the hypervisor (already done on our VMs): Strata's 29 GB n-gram table stalls on a disk it thinks is rotational
--low-ram resident is the setting that matters. The default (mmap) keeps re-reading the model from disk and loses ~90% of prompt speed on a V100 at 30 GB and a third of decode on a P40 at 40 GB
An 8 GB swap file: in resident mode the experts are ordinary process memory, and without swap a picture or a long prompt gets the engine killed instead of slowed down
Model download ~84 GB, resumable; loading takes 1–2 minutes

Tested and not recommended:

Two requests at once (parallel: 2): every request slower on the V100, including a lone one; queued requests finish sooner
Unsloth UD-Q4_K_XL (4-bit) on two P40s: ~25% slower decode and 80 GB of RAM for the same answers
ik_llama.cpp on a P40: ~8 tok/s
Setup's default --low-ram at 30–40 GB (mmap)

The server is OpenAI-compatible (/v1/chat/completions) and Anthropic-compatible (/v1/messages), so Open WebUI, LangChain, VS Code and Claude-style clients connect unchanged. Put a key in STRATA_API_KEY; without it the server answers 401.

P40:ECC must be off on the P40 (nvidia-smi -e 0, then reboot): with it on the card shows 23,040 MiB instead of 24,576. Engine builds for sm_61 through Strata's experimental Pascal path. 61 °C peak in our runs.

Strata v0.1.44.1 (commit 296b59d2), ISTA-DASLab GSQ-RCO IQ3_S, CUDA 12.8.93, driver 580.178.04, Ubuntu 26.04.1, AMD EPYC 7532 vCPUs. Prompt reading and decode are the engine's own timings, mean of 2 runs per size, 512 tokens out, thinking off.

Specced for Strata · measured on our own VMs, 17–19 Oct 2026

Qwen3.8-Flash-Next (125B) at 72–79 tokens a second, on one V100.

Qwen3.8-Flash-Next · 125B mixture-of-experts frontier model (6B active per word) · ISTA-DASLab GSQ-RCO IQ3_S · 128K context · Strata v0.1.44.1

72–79tok/swriting · Qwen3.8-Flash-Next · 128K context

~1,160–1,450tok/sreading a prompt

125BQwen3.8-Flash-Next, a frontier mixture-of-experts model, on one card

Qwen3.8-Flash-Next (125B), a frontier mixture-of-experts model, with a 128K context on one 32 GB card; 256K tested
Its 32 GB of HBM2 holds about half of the model's experts, so it needs less system RAM than the P40: this plan's 35 GB covers 128K text (measured at 30 GB); the 256K context and the picture encoder were measured at 36 GB, where the 8 GB swap file gives the headroom
Pictures: about 5.5 s each through the CPU encoder, with no measurable cost to text speed
Everything above 128K and every picture was measured on this card

What that means in practice (worked out from the measured speeds above):

For example, it can read a 50-page document (35K tokens) in about 27 seconds
For example, it can read a 130-page document (87K tokens) in about a minute
For example, it can read a 250-page document (162K tokens) in about 2 minutes 15 seconds with the 256K context
For example, at 75 tok/s it writes about 55 words a second

Measured speed by prompt size, 1 × V100: prompt reading / writing in tokens per second
Prompt	3.8K tokens	35K tokens	87K tokens	162K tokens
Reading	1,163 tok/s	1,301 tok/s	1,404 tok/s · at 40 GB RAM	1,228 tok/s · 256K context, 32 GB RAM
Writing	72 tok/s	75 tok/s	69 tok/s	62 tok/s

On the same model, llama.cpp needed three V100s to reach ~51 tok/s; Strata reaches 72–79 tok/s on one.

Pictures too, without slowing the text down.Strata reads pictures through an optional 0.9 GB encoder. Run it on the CPU: it needs no VRAM and costs no measurable text speed. A new picture takes 5.2–5.9 s with 6 threads; a picture sent again (chat apps resend it every turn) costs ~0.2 s. Receipts, bar charts and terminal screenshots were all read right: 36 of 36 answers over four runs. On the V100 the encoder can also run on the GPU (1.5–3.3 s per picture, 4–11% slower text).

What Strata needs on the VM, and what we tested and do not recommend

Ubuntu 26.04 (tested), NVIDIA driver 580, CUDA 12.8 toolkit (CUDA 13 dropped the P40; 12.9 does not build on this glibc), gcc 14
An SSD marked as such in the hypervisor (already done on our VMs): Strata's 29 GB n-gram table stalls on a disk it thinks is rotational
--low-ram resident is the setting that matters. The default (mmap) keeps re-reading the model from disk and loses ~90% of prompt speed on a V100 at 30 GB and a third of decode on a P40 at 40 GB
An 8 GB swap file: in resident mode the experts are ordinary process memory, and without swap a picture or a long prompt gets the engine killed instead of slowed down
Model download ~84 GB, resumable; loading takes 1–2 minutes

Tested and not recommended:

Two requests at once (parallel: 2): every request slower on the V100, including a lone one; queued requests finish sooner
Unsloth UD-Q4_K_XL (4-bit) on two P40s: ~25% slower decode and 80 GB of RAM for the same answers
ik_llama.cpp on a P40: ~8 tok/s
Setup's default --low-ram at 30–40 GB (mmap)

V100:Leave ECC on: on HBM2 it costs no memory. Engine builds for sm_70. 63 °C peak in our runs.

Pick a card

P40 or V100?

On the same models and prompts the V100 measured about 2.5× faster than the P40 on average. The P40 is the affordable private AI server; the V100 adds memory, tensor cores and long context. Same software on both.

	NVIDIA Tesla P40	NVIDIA Tesla V100
Strata · Qwen3.8-Flash-Next (125B), one card	39–41 tok/s writing · ~450 tok/s reading · 128K context	72–79 tok/s writing · ~1,160–1,450 tok/s reading · 128K context (256K tested)
Card memory	24 GB per card	32 GB HBM2 per card, 900 GB/s
Tensor cores	No	Yes (FP16)
Writing speed, one card	73 tok/s · Qwen3.6-35B-A3B, 64K context	154 tok/s · Qwen3.8-35B-A3B Distill, 128K context
Reading speed, one card	~1,040 tok/s	~2,300 tok/s
Longest context, one card	128K tokens (~200 pages) · Qwen3.8-35B-A3B Distill, up to 60 tok/s	262K tokens (~400 pages) · Qwen3.6-35B-A3B, up to 132 tok/s
A team on one card	4 people at 28 tok/s each (106 tok/s total)	8 people at ~33 tok/s each (~210 tok/s total)
Biggest model, three cards	Strata on one card: Qwen3.8-Flash-Next, 125B, 39–41 tok/s	Strata on one card: Qwen3.8-Flash-Next, 125B, 72–79 tok/s
Fine-tuning	QLoRA up to 9B, measured: 76 training tok/s	QLoRA up to 27B fits (26–28 GB); training speed being measured
Operating system	Linux or Windows. Driver preinstalled on both; training measured on Windows Server 2022.	Linux or Windows. Driver preinstalled on Ubuntu; all figures measured on Ubuntu.
One card, per month	$139CA$191 · P40 GPU Slice 1 - 1xP40 (24GB VRAM)	$179CA$247 · V100 GPU Slice 1 - 1xV100 (32GB VRAM)

NVIDIA Tesla P40 · 24 GB · full PCI passthrough

Qwen3.6-35B-A3B, a private API, 73 tokens a second, on one card.

Rent a VPS with a whole Tesla P40 passed through to it. Run open models, serve your team, read your documents and fine-tune on your own data. Nothing leaves the VM, and there is no per-token bill.

$ nvidia-smi --query

Tesla P40: passthrough · yours alone
memory: 22.2 / 24 GB
model: Qwen3.6-35B-A3B · UD-Q4_K_S
context: 64,000 tokens (~100 pages)
reads: ~1,040 tok/s
writes: 73 tok/s (~55 words/s)
cpu: almost idle, 2 vCPU is enough

More cards

With Strata, one card already runs the 125B model. Extra cards are for more output or two independent servers. Need more speed per card? The V100 is about 2.5× faster on average, with 32 GB.

1 × P40 73 tok/s

Qwen3.6-35B-A3B, a 35B mixture-of-experts model, with a 64K context (~100 pages)
Or a full 128K context (~200 pages) with Qwen3.8-35B-A3B Distill, up to 60 tok/s
Up to ~4 people at once: with -np 4 each gets 28 tok/s and the card's total output rises to 106 tok/s

P40 GPU Slice 1 - 1xP40 (24GB VRAM) · $139/moCA$191/mo →

2 × P40 148 tok/s

Two independent AI servers in one VM, each at full speed (0.4% interference)
Or Qwen3.8-35B-A3B Distill across both cards at up to 72 tok/s with its 128K window
Halves the wait for the first word when 4 people ask at once (26 s → 15 s)

P40 GPU Slice 2 - 2xP40 (48GB VRAM) · $278/moCA$383/mo →

3 × P40 · Qwen3.8-Flash-Next ~28 tok/s

A 125B frontier mixture-of-experts model (6B active per word), with vision
Measured over 14 hours of real work: 28 tok/s writing, ~140–215 tok/s reading
Native context up to 256K tokens; about 17 GB of the model sits in the plan's system RAM

P40 GPU Slice 3 - 3xP40 (72GB VRAM) · $417/moCA$575/mo →

The models that run best on 24 GB

With Strata, Qwen3.8-Flash-Next (125B) is the headline. With llama.cpp, mixture-of-experts models such as Qwen3.6-35B-A3B, which activate about 3B of their 35B parameters per word, fit one card and write faster than you read. Memory is what the job actually used on one card.

Model and setup	Card memory used	Context	Speed
Specced for · Strata Qwen3.8-Flash-Next (125B) Strata · ISTA-DASLab GSQ-RCO IQ3_S · 128K context · the whole card plus 40 GB of system RAM holding the model's experts.	whole card + 40 GB RAM	128K	39–41 tok/s writing
Our pick for documents Qwen3.8-35B-A3B Distill Newest generation · Q4_K_M · a full 128K context (~200 pages) on one card. Thinks briefly, then answers: a median of 5 seconds per answer on our 40-question document test.	23.1 GB	128K	up to 60 tok/s writing
Fastest writer Qwen3.6-35B-A3B Mixture of experts · UD-Q4_K_S · 64K context. The fastest writer on one card.	22.2 GB	64K	up to 73 tok/s writing
Fine-tuning Qwen3.5-9B (QLoRA) QLoRA (4-bit) in Unsloth Studio on Windows Server 2022.	13.8 GB	—	76 training tok/s

The Distill reads documents at about 950 tok/s, so a 15-page document is read in about 9 seconds. Both models carry a built-in accelerator (MTP) that lets them write several words per step.

Serving a team? With Qwen3.6-35B-A3B, add -np 4: four people typing at once each get 28 tok/s, and the card's total output rises to 106 tok/s.

Fine-tune in Unsloth Studio on Windows

Some teams would rather click than type. Our Windows Server 2022 image comes with the P40 driver installed and the current Visual C++ runtime that modern AI tools need. Install the Unsloth Studio desktop app, pick a model and a dataset, and train. We ran it end to end on this image: the loss fell, the result loaded and answered, and the card did the work.

model: Qwen3.5-9B-Base, QLoRA (4-bit)
dataset: gsm8k, 30 steps, 48,203 tokens
time: 17 min 26 s, completed
loss: ~0.6–0.78 → 0.46, falling
speed: ~76 training tok/s once warm (21 s/step)
memory: 13.8 of 24 GB, so 14B is likely next
one epoch: ~6 hours (estimated from the run)
also: Qwen3-0.6B LoRA, on the card at up to 232 W

On Windows the P40 is a compute card: CUDA, PyTorch and training, with no display acceleration.

What customers build on the P40

Ask your own documents

Load a contract, a manual or a year of notes and ask questions. With Qwen3.8-35B-A3B Distill the typical answer comes back in about 5 seconds.

A private ChatGPT for your team

Open WebUI in a second container, on your domain, with accounts. Company data stays on your server.

Coding assistant

Point VS Code or JetBrains extensions at your VM instead of a cloud API. The server speaks the OpenAI API.

Agents and automation

Models that call your own tools and APIs, with no rate limit and no per-call cost.

Images and screenshots

Qwen3.8-27B reads images and screenshots on one card, and Qwen3.8-Flash-Next does it on three, next to its text work.

Fine-tune on your data

LoRA and QLoRA up to 9B parameters. A few thousand examples train overnight, then the model is yours.

Figures: llama.cpp on Ubuntu, 1–2 × P40, 6 vCPU / 30 GB per card · Qwen3.8-35B-A3B Distill on llama.cpp b11459, one P40, 128K context, Oct 2026 · Qwen3.8-Flash-Next on 3 × P40, real workload, Oct 2026 · training on Windows Server 2022, Unsloth Studio, 13 Oct 2026.

NVIDIA Tesla V100 · 32 GB HBM2 · full PCI passthrough

Qwen3.8-35B-A3B Distill, a 128K context, 154 tokens a second, on one card.

Rent a VPS with a whole Tesla V100 passed through to it. 32 GB of fast HBM2 memory and tensor cores run large, high-precision models and read whole documents in seconds. Serve your team, build agents, keep your data private. Nothing leaves the VM, and there is no per-token bill.

$ nvidia-smi --query

Tesla V100: passthrough · yours alone
memory: 27.2 / 32 GB
model: Qwen3.8-35B-A3B Distill · Q4_K_M
context: 128,000 tokens (~200 pages)
reads: ~2,300 tok/s
writes: 154 tok/s (~115 words/s)
cpu: almost idle, 2 vCPU is enough

More cards

Team figures are Qwen3.6-35B-A3B on one card. Three-card figures are from production work on our own servers.

1 × V100 154 tok/s

Qwen3.8-35B-A3B Distill, a 35B mixture-of-experts model, with a 128K context (~200 pages)
Or the full 262K window (~400 pages), up to 132 tok/s
Or a 27B dense model at 8-bit
A team on one card: eight people at once get about 33 tok/s each (~210 tok/s total); four people get 57 tok/s each

V100 GPU Slice 1 - 1xV100 (32GB VRAM) · $179/moCA$247/mo →

2 × V100 2 servers

Not yet measured; per-card figures apply.

Run two separate model servers, one per card, each at the single-card figures on this page
Or split a model that needs more than 32 GB across the two cards (64 GB total)
We have not yet published a measured two-card V100 workload; every speed on this page is per card

V100 GPU Slice 2 - 2xV100 (64GB VRAM) · $358/moCA$494/mo →

3 × V100 · Qwen3.8-Flash-Next ~51 tok/s

A 125B frontier mixture-of-experts model (6B active per word)
The whole model sits in 96 GB of card memory, with a 128K context
Measured over hours of real work: ~51 tok/s writing, ~370 tok/s reading

V100 GPU Slice 3 - 3xV100 (96GB VRAM) · $537/moCA$741/mo →

The models that run best on 32 GB

With Strata, Qwen3.8-Flash-Next (125B) is the headline. With llama.cpp, the extra room is what makes this card: higher-precision builds, long context windows and large dense models fit on one card at full speed. Memory is what the model actually used on one card.

Model and setup	Card memory used	Context	Speed
Specced for · Strata Qwen3.8-Flash-Next (125B) Strata · ISTA-DASLab GSQ-RCO IQ3_S · 128K context · the whole card plus 35 GB of system RAM holding the model's experts.	whole card + 35 GB RAM	128K	72–79 tok/s writing
Our pick for documents Qwen3.8-35B-A3B Distill Newest generation · mixture of experts · Q4_K_M · a full 128K context (~200 pages). Reads about 2,300 tokens a second.	27.2 GB	128K	up to 154 tok/s writing
Higher precision Qwen3.8-35B-A3B Distill, 5-bit Q5_K_M · the same model at higher precision, still with a 128K context.	30.6 GB	128K	up to 150 tok/s writing
Longest context Qwen3.6-35B-A3B Mixture of experts · UD-Q4_K_M · the model's full 262K window (~400 pages) at full speed.	30.3 GB	262K	up to 132 tok/s writing
Dense, 8-bit Qwen3.8-27B Dense · Q8 · the highest practical precision for a 27B model, with its built-in accelerator.	31.5 GB	32K	up to 47 tok/s writing
Dense, 4-bit Qwen3.8-27B Dense · 4-bit with its built-in accelerator · reads a 10-page document in about 10 seconds.	20.2 GB	32K	up to 59 tok/s writing
Dense, 4-bit Muse Glimmer 30B Dense · 4-bit with its DFlash draft model.	19.5 GB	32K	up to 60 tok/s writing

Mixture-of-experts models use only about 3B of their 35B parameters per word, which is why they write so fast. Dense models use every parameter, so they write slower; the V100's tensor cores still read their prompts at 750–1,000 tokens a second.

Serving a team? Add -np 8: with Qwen3.6-35B-A3B, eight people typing at once each get about 33 tok/s, and the card's total output reaches about 210 tok/s. Four 10-page documents, all answered in 38 seconds.

Fine-tune a 27B model on one card

Teach a model your own style, terminology or task. With 32 GB, the V100 has room to fine-tune larger models than a 24 GB card can: QLoRA training of a 27B model such as Qwen3.8-27B needs about 26–28 GB, and fits on one V100. Unsloth Studio, the desktop training app, makes it point-and-click: pick a model, load your dataset, press train. Your data and the trained model never leave your VM.

tool: Unsloth Studio, desktop app
method: LoRA and QLoRA (4-bit)
largest: Qwen3.8-27B, about 26–28 GB of 32 GB
smaller: 9B models and below, with room for longer training examples
precision: FP16 on tensor cores
speed: being measured

We are measuring training on the V100 now and will publish the figures here. The V100 trains in FP16 (it has no BF16), which Unsloth supports.

A whole book in one prompt

With 32 GB, the V100 keeps a 128K or even 262K context on the card, so the model reads the whole document at once instead of in pieces, and answers about all of it. We measured it with a real 82,000-token document, about 120 pages.

document: 82,000 tokens, ~60,000 words, ~120 pages
model: Qwen3.8-35B-A3B Distill, 128K context
read in: 57 seconds (~1,420 tok/s)
answer: 113 tok/s, with the whole document in context
memory: 27.2 of 32 GB
longest: 262K tokens (~400 pages) with Qwen3.6-35B-A3B, 30.3 GB, 132 tok/s
clock: full speed throughout, never throttled (peak 151 W, 50 °C)

Reading speed on mixture-of-experts models comes from our V100-tuned build of llama.cpp (see Setup). The stock build reads about half as fast.

What customers build on the V100

Ask your own documents

Contracts, manuals, codebases and long chat histories, up to ~400 pages in one prompt. A 120-page document is read in under a minute.

A private ChatGPT for your team

Open WebUI on your domain, with accounts. Eight people can use one card at once, each faster than they can read.

Higher-quality models

Dense 27B models at 8-bit and 35B models at 5-bit, on a single card at full speed.

Coding assistant

Point VS Code or JetBrains extensions at your VM instead of a cloud API. Whole files and long diffs fit in the context.

Agents and automation

Models that call your own tools and APIs, with no rate limit and no per-call cost. At 150 tok/s, multi-step chains stay quick.

Very large models

Combine cards for models that need more than 32 GB, such as Qwen3.8-Flash-Next (125B) with llama.cpp on three V100s, or run Strata on each card.

Figures: llama.cpp b11580 on Ubuntu 26.04, one Tesla V100-PCIE-32GB per VM, 14–15 Oct 2026 · Qwen3.8-Flash-Next on 3 × V100, real workload, Oct 2026 · on the same models and prompts the V100 measured about 2.5× faster than a Tesla P40 on average · full benchmark report on request.

Set it up

Two ways to run it: Strata or llama.cpp

Strata for Qwen3.8-Flash-Next (125B) on one card, the model these plans are specced for. llama.cpp for the smaller 35B and 27B models at higher speed. Both serve the OpenAI API, so Open WebUI, VS Code and LangChain connect the same way. Pick a tab.

Strata: Qwen3.8-Flash-Next (125B) on a P40 or V100, step by step

The exact steps we measured with, on our own VMs (Ubuntu 26.04.1, kernel 7.0, glibc 2.43, NVIDIA driver 580.178.04, Strata v0.1.44.1). Steps are the same for both cards except where a P40 or V100 label says otherwise. Loading takes 1–2 minutes once installed.

The model files take 78 GB and Strata's prepared copy 56 GB, plus ~3 GB for CUDA, the 8 GB swap file and the OS. The disk must be an SSD and marked as one in the hypervisor (already done on our VMs: ssd=1, discard=on); otherwise Strata warns that its 29 GB n-gram table will stall prompts.

Hold the driver packages

The driver is already installed. Hold its packages once it works: an unattended upgrade that moves the libraries but not the loaded module stops every new GPU process with "Driver/library version mismatch".
```
apt-mark hold $(dpkg -l | awk '/^ii/ && $2 ~ /nvidia/ {print $2}')
```
P40 ECC: off on the P40, untouched on the V100

A P40 with ECC on shows 23,040 MiB instead of 24,576. Turn it off and reboot, then check. On a V100 leave ECC as it is: on HBM2 it costs no memory (32,768 MiB either way).
```
nvidia-smi -e 0 && reboot
nvidia-smi --query-gpu=name,memory.total --format=csv   # after the reboot: 24576 MiB
```
Add an 8 GB swap file

In resident mode the model's experts are ordinary process memory. Without swap, a picture or a long prompt that needs a little more gets the engine killed instead of slowed down. 8 GB covers the largest spike we saw (1.8 GB) more than four times over; with the CPU image encoder it peaked at 0.2 GB.
```
fallocate -l 8G /swapfile && chmod 600 /swapfile && mkswap /swapfile && swapon /swapfile
cp -p /etc/fstab /etc/fstab.pre-swap && echo '/swapfile none swap sw 0 0' >> /etc/fstab
```
Install gcc 14

CUDA 12.8 refuses gcc 15, which is Ubuntu 26.04's default.
```
apt-get install -y gcc-14 g++-14 git
```

Install the CUDA 12.8 toolkit (toolkit only; the driver stays)

Use 12.8: CUDA 13 has no Pascal (P40) support, and 12.9 does not compile on glibc 2.43. On Ubuntu 26.04 the runfile's installer needs libxml2.so.2, which is gone, so unpack it and copy the parts Strata needs. On Ubuntu 24.04 (glibc 2.39) the normal --silent --toolkit install should work; we did not test it.

cd /root && curl -fLO https://developer.download.nvidia.com/compute/cuda/12.8.1/local_installers/cuda_12.8.1_570.124.06_linux.run
sh cuda_12.8.1_570.124.06_linux.run --noexec --keep --target /root/cuda-x
B=/root/cuda-x/builds; T=/usr/local/cuda-12.8; mkdir -p $T
for c in cuda_nvcc cuda_cudart cuda_cccl cuda_nvrtc cuda_nvml_dev cuda_cuobjdump cuda_cuxxfilt \
         cuda_nvtx cuda_profiler_api cuda_nvprune cuda_nvdisasm libcublas libcurand; do cp -a $B/$c/. $T/; done
$T/bin/nvcc --version | tail -1

Fix six declarations in the CUDA math header

They clash with glibc 2.43 (needed for 12.8 too, despite Strata's docs). The original is kept as .orig.

F=/usr/local/cuda-12.8/targets/x86_64-linux/include/crt/math_functions.h; cp -n $F $F.orig
sed -i -E 's/(rsqrtf?|sinpif?|cospif?)\((double|float) x\);/\1(\2 x) noexcept (true);/' $F
diff $F.orig $F | grep -c '^>'     # expect 6

Install Strata and let its setup build the engine for your card

Use the release we measured. Setup downloads the model (~84 GB, resumable), prepares it, fetches the MTP draft layer and writes strata-iq3_s.json and run-iq3_s.sh. The engine builds for sm_61 on a P40 (Strata's experimental Pascal path) and sm_70 on a V100; if you ever move the VM between card types, re-run this command. Always pass --low-ram resident: at 30–40 GB of RAM the default (mmap) keeps re-reading the model from disk and loses ~90% of prompt speed on a V100 and a third of decode on a P40. Check that the config's args contain --resident-experts.
```
git clone https://github.com/Niko1221/Strata.git /opt/strata/src
cd /opt/strata/src && git checkout v0.1.44.1
export STRATA_NVCC=/usr/local/cuda-12.8/bin/nvcc PATH=/usr/local/cuda-12.8/bin:$PATH \
       CC=gcc-14 CXX=g++-14 CUDAHOSTCXX=g++-14
./setup.sh --setup --build --cuda 12 --model IQ3_S --context 131072 \
           --low-ram resident --vision no --images no \
           --host 0.0.0.0 --data-dir /opt/strata/data --yes --no-start --no-browser
grep -c resident-experts /opt/strata/src/strata-iq3_s.json   # expect 1
```
Start it with an API key

Strata takes its key from STRATA_API_KEY. Keep it in a root-only file rather than on the command line or in the config. Loading takes 1–2 minutes; without the key the server answers 401. The API is OpenAI-compatible (/v1/chat/completions) and Anthropic-compatible (/v1/messages); any model name works.
```
umask 077; python3 -c 'import secrets; print("sk-" + secrets.token_urlsafe(24))' > /root/strata-key.txt
STRATA_API_KEY=$(cat /root/strata-key.txt) /opt/strata/src/run-iq3_s.sh
curl -s -H "Authorization: Bearer $(cat /root/strata-key.txt)" http://127.0.0.1:8080/v1/models
```

Run it as a service

We started run-iq3_s.sh by hand or from llama-swap during the measurements; this unit is the straightforward way to keep it up.

# /etc/systemd/system/strata.service
[Unit]
Description=Strata (Qwen3.8-Flash-Next IQ3_S)
After=network-online.target

[Service]
ExecStart=/bin/sh -c 'STRATA_API_KEY=$(cat /root/strata-key.txt) exec /opt/strata/src/run-iq3_s.sh'
Restart=on-failure
RestartSec=10

[Install]
WantedBy=multi-user.target

Pictures (optional): run the encoder on the CPU

Re-run setup with --vision cpu, then in strata-iq3_s.json set "threads" in the "vision" section to the VM's core count (setup gives it only half) and restart Strata. A new picture then takes about 5–6 s with no measurable cost to text speed; a resent picture ~0.2 s. Send pictures as an OpenAI image_url part (a data: URL works). On a V100 the encoder can also run on the GPU (--vision gpu: 1.5–3.3 s per picture, 4–11% slower text). The on-demand "elastic" GPU mode does not work with resident mode. Images on a P40 were not measured; plan about 2 GB more RAM.
```
cd /opt/strata/src   # with the same export line as above
./setup.sh --setup --build --cuda 12 --model IQ3_S --context 131072 \
           --low-ram resident --vision cpu --images no \
           --host 0.0.0.0 --data-dir /opt/strata/data --yes --no-start --no-browser
```

Health checks

The dmesg line should print nothing: a killed process means the engine ran out of memory.

nvidia-smi --query-gpu=name,memory.used,memory.total,temperature.gpu --format=csv
free -h; swapon --show
dmesg -T | grep -i "killed process"          # should print nothing
grep -c resident-experts /opt/strata/src/strata-iq3_s.json   # expect 1

More than one card. Two P40s: pass --gpus 0,1 to setup and it splits the layers across the cards (measured with 12 vCPU and 60 GB RAM). Two V100s were not measured.

llama.cpp: running in about five minutes

The model server is llama.cpp in Docker, serving a GGUF file over the OpenAI API. Download the model once, start the container, point your tools at port 8080. The exact commands we measured with are below, one per card.

Order a plan and pick Ubuntu at deployment. The NVIDIA driver is already installed.
Add Docker and the NVIDIA Container Toolkit so containers can see the card.
Download a model (about 21 GB for the 35B mixture-of-experts builds) into /opt/models.
Start the server with the command for your card. The flags we list are the ones that measurably mattered.
Connect Open WebUI, VS Code, LangChain or anything else that speaks the OpenAI API, behind your own proxy with authentication.

Setup on Ubuntu: running in about five minutes (P40)

On Ubuntu the driver is already there. Add Docker and the NVIDIA Container Toolkit, download the model once (21.4 GB), then start the server. Nothing to compile.

mkdir -p /opt/models && curl -L -o /opt/models/Qwen3.6-35B-A3B-UD-Q4_K_S.gguf \
  https://huggingface.co/unsloth/Qwen3.6-35B-A3B-MTP-GGUF/resolve/main/Qwen3.6-35B-A3B-UD-Q4_K_S.gguf
docker run -d --name llm --gpus all -v /opt/models:/models -p 127.0.0.1:8080:8080 \
  ghcr.io/ggml-org/llama.cpp:server-cuda-b10438 \
  -m /models/Qwen3.6-35B-A3B-UD-Q4_K_S.gguf \
  --host 0.0.0.0 --port 8080 \
  -c 65536 -np 1 -t 6 -tb 6 -b 4096 -ub 2048 \
  -ngl auto --fit on --fit-target 1024 \
  --spec-type draft-mtp --spec-draft-n-max 2 --jinja

-ub 2048: Reads documents 29% faster. Use it on single-card setups.
--spec-draft-n-max 2: Turns on the model's built-in accelerator: +45% writing speed. 2 beat 1, 3 and 4.
-m /models/…gguf: Serve the file by path. The -hf shortcut also loads a vision add-on that costs 18% writing speed. The image is pinned to the build we measured.

The server speaks the OpenAI API on 127.0.0.1:8080/v1: LangChain, LlamaIndex, Open WebUI and Continue work unchanged. Keep it behind your own proxy with authentication.

Setup on Ubuntu: running in about five minutes (V100)

On Ubuntu the driver is already there. Add Docker and the NVIDIA Container Toolkit, download the model once (21.7 GB), then start the server.

mkdir -p /opt/models && curl -L -o /opt/models/Qwen3.8-35B-A3B-Distill-Q4_K_M.gguf \
  https://huggingface.co/empero-ai/Qwen3.8-35B-A3B-Distill-GGUF/resolve/main/Qwen3.8-35B-A3B-Q4_K_M.gguf
docker run -d --name llm --gpus all -v /opt/models:/models -p 127.0.0.1:8080:8080 \
  ghcr.io/ggml-org/llama.cpp:server-cuda-b11580 \
  -m /models/Qwen3.8-35B-A3B-Distill-Q4_K_M.gguf \
  --host 0.0.0.0 --port 8080 \
  -c 131072 -np 1 -t 6 -tb 6 -b 4096 -ub 4096 \
  -ngl auto --fit on --fit-target 1024 \
  --spec-type draft-mtp --spec-draft-n-max 2 --jinja

-ub 4096: A larger reading batch: +27% reading speed. The 32 GB has room for it.
--spec-draft-n-max 2: Turns on the model's built-in accelerator: +47% writing speed. 2 beat 1, 3 and 4.
GGML_CUDA_FORCE_MMQ: One build option of llama.cpp doubles reading speed on mixture-of-experts models (the ~2,300 tok/s above). Keep the stock image for dense models. We share the build recipe.

The server speaks the OpenAI API on 127.0.0.1:8080/v1: LangChain, LlamaIndex, Open WebUI and Continue work unchanged. Keep it behind your own proxy with authentication.

Plans

6 GPU VPS plans

Each plan page shows the figures for its own card count: the 3-card plans show the 125B-model numbers, the 1-card plans the single-card tables.

1 × NVIDIA Tesla P40 24 GB AMD EPYC Gen 2 / Intel Xeon Gold Gen 2 10 vCPU 40 GB 500 GB NVMe

High0.5–4TB>$30

VDS — Dedicated Cores NVMe Storage VPS Large NVMe Storage