llm-box-uk

Let me check the build catalogue to see what the Bosgame build was and what the alternatives are.

• Read /home/megaproxy/claude/research/llm-box-uk/memory.md 11ms 7.7 KB

• Bash grep -in "bosgame\|strix\|halo\|395\|mac studio\|epyc\|unified" index.html | head -60 103ms 6.1 KB

• Bash grep -n "recommend\|verdict\|headline\|pick\|Build F\|3090" index.html | head -40 25ms 7.1 KB

Let me look at the recommendation/verdict section to see the actual "top picks."

Right — "the Bosgame one" is the **Strix Halo / Ryzen AI Max+ 395 mini-PC class** (Build **L** in the catalogue, where I'd priced it as the GMKtec EVO-X2; Bosgame sells the same 395 chip in the same shape). It's the cheapest road to 128GB unified, but yes — it's a no-name Chinese mini-PC brand with the warranty/longevity question that implies.

The catalogue framed the "one complete quiet box that holds big models" decision as a three-way (K / L / M). With L off the table, the other top choices were:

| Pick | Box | Capacity / speed | Price (June 2026) | Why it's the alternative |
|---|---|---|---|---|
| **K** | **Mac Studio M1 Ultra 128GB** | 128GB unified @ **800 GB/s** | ~£1,950 used | The genuine "I don't trust a Chinese mini-PC" upgrade. Apple build quality, silent, bandwidth in a 3090's league but 5× the capacity. Runs 70B fast, 120B at Q4. macOS/MLX only, no CUDA. |
| **M** | **EPYC 7532 + 384GB DDR4** | 384GB system RAM @ ~200 GB/s | ~£1,375 | The CPU route to giant **MoE** models — 235B-A22B at ~5–10 t/s with no GPU. Dense models crawl. Add a 3090 + ktransformers → ~15–20 t/s. |
| **F / H** | **RTX 3090 (single / dual)** | 24GB / 48GB VRAM | £1,210 / £1,965 | The discrete-GPU pick and the only route that also does **fine-tuning** properly — full CUDA, QLoRA, longest software runway. Brand-name cards, no import-VAT gamble. |

The verdict's one-line summary: **F (single 3090)** if you want one box that does everything including training; **K (Mac Studio)** if you want the quiet, large-capacity, non-Chinese inference box that the Strix Halo was trying to be.

---

One thing worth flagging, though: **£6,000 at 0% for 25 months changes the math.** Every build in the catalogue tops out at ~£1,965 — you've got ~3× that to spend, interest-free. That genuinely reopens the top end:

- **Mac Studio M3 Ultra 256GB / 512GB** (~819 GB/s, current-gen, in budget) — a different league from the used M1 Ultra.
- **2–4× RTX 3090** (96GB VRAM) or a **single RTX 6000-class** card for real fine-tuning headroom.
- **EPYC + 512GB** for DeepSeek-V3-class MoE.

Want me to re-spec the top picks against the actual £6,000 ceiling (and re-check current prices), rather than the ~£2k-ceiling builds in the catalogue? The doc was costed for value-per-pound, not for "I have £6k of free money."

Good call — staying ~£3–3.5k still triples the catalogue's ceiling, so each of the three routes I just listed gets meaningfully better without you blowing the whole card. Here's how they scale to that budget (rough June-2026 prices — I'd want to live-check before you buy):

**Route 1 — NVIDIA, 3× RTX 3090 (72GB VRAM)** · ~£3,000–3,200
- 3× used 3090 @ ~£780 = £2,340, + a proper host (Threadripper/EPYC or X99, 1500W PSU, open frame, shrouds) ~£600–800.
- **The only route that does your fine-tuning leg properly** — full CUDA, QLoRA on 70B, longest software runway. Runs 70B comfortably, 120B at Q4. Loud and power-hungry, but that's exactly what the garage/rack location is for.

**Route 2 — Mac Studio M2 Ultra 192GB (used)** · ~£3,200–3,800
- Quiet, premium, genuinely *not* a sketchy Chinese mini-PC — the real version of what the Strix Halo box was reaching for. 192GB unified @ ~800 GB/s.
- Inference king (120B+ at good quant, MoE too). **But fine-tuning is MLX-light only** — no CUDA, no bitsandbytes/Unsloth. If the LoRA leg matters, this loses.

**Route 3 — EPYC + 512GB + a single 3090 (MoE box)** · ~£2,800–3,200
- 512GB DDR4 + ktransformers offload to one 3090 → DeepSeek-V3 / Qwen-235B-class MoE at ~15–20 t/s. Niche — only if running giant MoE models is the actual goal. Dense models still crawl.

**My pick, given your original brief (inference + light LoRA/QLoRA):** Route 1. The fine-tuning requirement is the whole reason the catalogue landed on the 3090 in the first place — at £3k you just get three of them instead of one, and a 72GB CUDA box is a serious do-everything machine. Route 2 is the answer *only* if you've quietly decided this is really an inference-only box and you want silence over training.

Want me to pull live prices and write up a full BOM for one of these (slotted into `index.html` as a new £3k tier)? If so — is the fine-tuning leg still a real requirement, or has this drifted toward inference-first? That's the fork that decides between Route 1 and Route 2.

That pivot makes sense — and it quietly answers my fork question: you're prioritising *running* large models over the fine-tuning leg, which means unified memory is genuinely back on the table. The thing you didn't like about the Bosgame box was the *brand*, not the *idea*. So here are the two ways to get a single 128GB all-in-one box without the no-name-Chinese problem:

**Option 1 — Framework Desktop (Ryzen AI Max+ 395, 128GB)** · ~£2,000
- This is the key one: it's the **exact same Strix Halo chip** as the Bosgame/GMKtec box, but built by **Framework** — a US company, repairable-by-design, real warranty and support, proper Western brand. It's a tidy mini-ITX x86 PC (Windows or Linux), not a mystery mini-PC.
- It fixes your objection completely while keeping everything you liked. Leaves ~£1.5k of the card unspent.
- **The catch is the chip, not the brand:** ~256 GB/s memory bandwidth. Great for MoE models (235B-class) and mid-size, but *dense* 70B is slow (~5–8 tok/s). That limit is identical on the Bosgame — it's the silicon.

**Option 2 — Mac Studio M2 Ultra 128GB (used)** · ~£2,800–3,200
- Same all-in-one, 128GB, single quiet box idea — but **800 GB/s bandwidth, ~3× the Framework.** That's the number that actually decides whether running a 70B+ is pleasant vs painful. Premium Apple build, silent, genuinely not Chinese.
- macOS + MLX/llama.cpp Metal only. Stretch to **192GB (~£3.2–3.8k)** if you want 120B-class headroom.

**My take:** if "run large models" means you want them to actually run *fast*, the Mac Studio M2 Ultra is the box — bandwidth is the whole game for dense models, and the Framework will frustrate you on anything 70B dense. If you specifically want a real **x86 Windows/Linux PC** and you're mostly running MoE models (where 256 GB/s is fine), the Framework Desktop is the honest, trustworthy version of the box you originally liked — at £2k it's almost a no-brainer.

Two quick things before I build a proper BOM and price it live:
1. **macOS or x86 (Windows/Linux)?** That's the real fork — it picks Mac vs Framework outright.
2. Roughly what are you running — mostly **dense models** (Llama/Qwen 70B-class) or are you happy living in **MoE land** (DeepSeek, Qwen-235B)? Dense → Mac; MoE-mostly → Framework is fine and saves you £1k+.

Want me to spec whichever you lean toward into `index.html` as a new £3k all-in-one tier with current prices?

• ToolSearch select:WebSearch,WebFetch 5ms 100 B

You're right on both counts — Linux takes the Mac out entirely (no macOS = Option 2 is dead), and I shouldn't be quoting prices from memory on a real purchase. Let me actually verify current prices. Let me load web tools and check.

• WebSearch Framework Desktop 128GB Ryzen AI Max+ 395 price UK 2026 6.6s 2.8 KB

Now let me check live prices for the Linux-friendly 128GB boxes — Framework Desktop and the trustworthy-brand alternatives (HP makes a Strix Halo workstation too).

• WebSearch HP Z2 Mini G1a Ryzen AI Max 395 128GB price UK 6.6s 2.8 KB

• WebSearch best Ryzen AI Max+ 395 128GB mini PC Linux LLM 2026 Framework vs 8.6s 2.8 KB

Mac's moot now anyway (Linux), so let me get solid UK prices on the x86 options. Let me confirm Framework's UK pricing directly.

• WebFetch https://frame.work/gb/en/desktop 6.3s 570 B

• WebFetch https://box.co.uk/a40q1et-hp-z2-mini-g1a-copilot-pc-amd-ryzen-ai-max 2.5s 215 B

• WebFetch https://frame.work/gb/en/products/desktop-diy-amd-aimax300/configuration/new 6.6s 458 B

• WebSearch "Z2 Mini G1a" 128GB "£" price box.co.uk OR scan.co.uk OR currys 395 8.8s 3.4 KB

Good thing we checked — the live prices moved meaningfully from what I'd quoted, and Mac's irrelevant now anyway. Here's what's actually true for a **Linux, single-box, 128GB, trustworthy-brand** machine today. Both real options are the *same Strix Halo chip* as the Bosgame — just from Western companies with real warranties:

| Box | Brand | Spec | Live UK price | Notes |
|---|---|---|---|---|
| **HP Z2 Mini G1a** | HP (enterprise) | Ryzen AI Max+ PRO 395 · 128GB · 1–2TB | **£2,397–2,664** | 3-yr warranty, UK retail stock (HP UK, Currys Business, Misco). The "definitely not sketchy" pick. |
| **Framework Desktop** | Framework (US) | Ryzen AI Max+ 395 · 128GB | **~£2k US ($1,999)… but the GB config page quoted me £3,449** | Best Linux support, repairable, mainboard-only option. The UK price needs nailing down — see caveat. |

**Price honesty:** I trust the HP number — it's anchored across multiple UK retailers ([Currys Business](https://business.currys.co.uk/catalogue/computing/desktops/windows-desktops/hp-workstation-z2-g1a-mini-ryzen-ai-max-pro-395-3-ghz-128-gb-ssd-1-tb-uk/P313057P), [box.co.uk](https://box.co.uk/a40q1et-hp-z2-mini-g1a-copilot-pc-amd-ryzen-ai-max), [VideoCardz launch](https://videocardz.com/newz/hp-z2-mini-g1a-amd-ryzen-ai-max-pro-compact-workstation-launches-in-uk-starting-at-1500)). The Framework one I do *not* fully trust — US is $1,999 for 128GB but [the UK config page](https://frame.work/gb/en/products/desktop-diy-amd-aimax300/configuration/new) read back £3,449 to me, which is a suspicious gap (VAT alone doesn't explain it). Could be a fetch misread or a UK markup; I'd verify on the live configurator before believing either number.

**The thing that doesn't change with brand — bandwidth.** Both are 256 GB/s, identical to the Bosgame, because it's the same silicon. So real performance ([benchmarked this month](https://runaihome.com/blog/mini-pc-local-llm-2026/)):
- **Dense 70B (Llama/Qwen) Q4: ~4–8 tok/s** — runs, but not snappy.
- **MoE — Qwen3-235B: ~11 tok/s**, gpt-oss-120B fine ([HP itself runs 120B on it with no GPU](https://www.storagereview.com/review/hp-z2-mini-g1a-review-running-gpt-oss-120b-without-a-discrete-gpu)).

That's the honest catch with *any* 128GB single-box at this price: MoE models fly, dense large models are usable-but-slow. No amount of brand-swapping fixes it — only the Mac Ultra (800 GB/s, but no Linux) or a multi-3090 rig (but not all-in-one) does.

**My recommendation:** the **HP Z2 Mini G1a 128GB at ~£2.4k**. It's the exact box you liked, on Linux, from an enterprise brand with a 3-year warranty and UK stock — and it leaves ~£1k of the card unspent. Only pick the Framework over it if its UK price actually lands near £2k *and* you value the repairability/openness.

Want me to verify the Framework UK price properly and then write up the HP as a new "£2.4k Linux all-in-one" tier in `index.html`? And one check: are you mostly running **MoE models** (then this class is genuinely great) or chasing **dense 70B speed** (then we should talk before you buy)?

You're right to call me out — I relayed prices from search snippets without verifying them against the live pages, and that's on me. £2,397 clearly isn't what you're seeing. Let me actually pull the real pages this time.

• WebFetch https://business.currys.co.uk/catalogue/computing/desktops/windows-desktops/hp-workstation-z2-g1a-mini-ryzen-ai-max-pro-… 3.6s 215 B

• WebFetch error https://www.hp.com/gb-en/shop/product.aspx?id=8T218EA&opt=ABU&sel=DTP 56.9s 27 B

• WebFetch https://www.gmktec.com/products/amd-ryzen%E2%84%A2-ai-max-395-evo-x2-ai-mini-pc 6.0s 608 B

Currys Business 403'd me and HP timed out, so I still can't see what you're seeing directly. Let me pin down the GMKtec UK price and dig into why the HP is showing £8k.

• WebFetch https://www.gmktec.uk/products/gmktec-evo-x2-amd-ryzen%e2%84%a2-ai-max-395-ai-mini-pc 6.1s 644 B

• WebSearch HP Z2 Mini G1a 128GB price review notebookcheck OR servethehome how much actual 10.5s 2.6 KB

Let me give you the straight version — and own that my earlier £2,397 was just wrong. I pasted it from a search snippet without loading the page, exactly the thing your CLAUDE.md warns against. Here's what I can now actually verify:

**Confirmed prices:**
- **GMKtec EVO-X2, 128GB + 2TB: £1,619.99** — direct from [gmktec.uk](https://www.gmktec.uk/products/gmktec-evo-x2-amd-ryzen%e2%84%a2-ai-max-395-ai-mini-pc), GBP inc VAT. Solid.
- **HP Z2 Mini G1a, 128GB:** HP's own list is **~$4,586** US ([Notebookcheck](https://www.notebookcheck.net/HP-Z2-Mini-G1a-with-AMD-Strix-Halo-review-Compact-workstation-with-Ryzen-AI-Max-and-Radeon-RX-8060S.1069652.0.html)). The **£8,049.99 ex-VAT** you're seeing on Currys Business is a padded B2B list price (Currys Business is notorious for this) — but even the *sane* HP street price is roughly **£3,500–4,500 inc VAT**. So it's expensive no matter how you slice it.
- **Framework Desktop, 128GB:** US is $1,999, but the GB configurator read back **£3,449** to me, which doesn't reconcile with the US price even after VAT. I genuinely don't trust either number and won't quote it as fact until I load the live cart — flagging that honestly rather than guessing again.

**Now, the actual answer to "how does it compare to GMKtec":**

**Performance: identical. Zero difference.** The HP, the Framework, the GMKtec and the Bosgame are all the *same* Ryzen AI Max+ 395, the same 128GB, the same 256 GB/s bandwidth, the same Radeon 8060S iGPU. Every one of them runs Llama-70B Q4 at ~4–8 tok/s and Qwen3-235B MoE at ~11 tok/s. The HP gives you **not one extra token/sec** over the GMKtec.

What the 2–3× premium actually buys you:
- HP: 3-year enterprise warranty/support, better-engineered cooling (quiet mode), build quality, resale value.
- Framework: US company, repairability, best-in-class Linux support, openness.
- GMKtec/Bosgame: ...the same compute, for £1,620, minus the warranty confidence and support.

So here's the uncomfortable truth your gut was circling: for an LLM box, "too Chinese" is a **£2,000–6,000 surcharge for a Western logo and a warranty on hardware that performs identically.** That's a lot to pay for peace of mind on a garage machine.

My honest read:
1. **If brand/warranty genuinely matters to you** → **Framework Desktop** is the cheapest non-Chinese route and has the best Linux story — *once I confirm its real UK price.* HP only if you want enterprise onsite warranty and accept paying the most.
2. **If you can stomach it** → buy the **GMKtec (or Bosgame) at ~£1,620**, ideally **through Amazon UK** so you get UK consumer-rights returns as a backstop, and pocket the £2k+. It's the rational value pick and it does everything the HP does.

Want me to load the live Framework UK configurator and get its real base price so you've got a clean three-way (GMKtec £1,620 / Framework £? / HP £3.5–4.5k) before you decide? And I should fix the bad HP figure in `index.html` while I'm at it.

Right, and that small Amazon premium is actually the most sensible money in this whole conversation — it's *exactly* buying down the thing you were worried about.

Think about what the gap pays for:
- **Direct from gmktec.uk (£1,619.99):** cheapest, but if it arrives faulty or dies in 6 months you're dealing with a Chinese vendor's returns process — the precise "too Chinese" risk that started this.
- **Amazon UK (a bit more):** same box, same chip, same performance, but now you've got Amazon's A-to-z guarantee, frictionless 30-day returns, and UK consumer rights enforced against a UK-facing entity. If it's a dud, you click "return" and a courier collects it.

So that delta isn't a markup you're losing — it's the **cheapest insurance policy available** against the one downside of the Chinese box. Paying ~£50–150 on Amazon to de-risk a £1,600 purchase from an overseas seller is a no-brainer compared to paying **£2,000–3,000 extra** for an HP/Framework badge that runs the models at the identical speed.

That actually resolves the whole thread cleanly:
- You wanted Linux, all-in-one, 128GB, big models — the Strix Halo box delivers it.
- You didn't want "too Chinese" — buying it **through Amazon UK** gives you Western buyer protection without the Western-brand tax.
- It keeps you well under your £3–3.5k ceiling (~£1,700-ish vs £3,500–8,000), leaving most of the 0% card untouched.

What's the Amazon price you're seeing? If you tell me (or I can pull the live listing), I'll lock down the real numbers and update `index.html` — both fixing my bad HP figure and adding GMKtec's "direct vs Amazon" pricing so the build catalogue reflects the actual decision you're making.

That changes things — and it tells me my £1,619.99 was almost certainly the **base/64GB** config, not the 128GB. The fetch flagged that it couldn't separate the per-config prices, and I should've caught it. The real 128GB GMKtec is what you're seeing: **£2,659.99 direct / £3,099.96 Amazon.** I'll trust your real-world numbers over my flaky fetches.

And that **demolishes the argument I just made you.** Here's the corrected picture for 128GB Strix Halo boxes:

| Box | Price | Brand |
|---|---|---|
| GMKtec **direct** | £2,659.99 | Chinese, China-direct returns |
| GMKtec **Amazon** | £3,099.96 | Chinese, but UK buyer protection |
| Framework Desktop | ~£3,449 *(still need to confirm)* | US, best Linux, repairable |
| HP Z2 Mini G1a | ~£3,500–4,500 street | Enterprise, 3yr warranty |

The "£2,000–6,000 brand tax" I claimed **doesn't exist** — at 128GB the gap is only a few hundred quid. That flips the logic entirely:

- The **£440 Amazon premium** isn't "a bit more" — it's nearly the whole step to a *Western brand*. Paying £3,099.96 for the GMKtec on Amazon is arguably the worst square on the board: you're at near-Framework money but still holding the Chinese box.
- For roughly **£350 more than the Amazon GMKtec**, Framework (if ~£3,449 is real) gets you an actual US company, the best Linux support in the category, and repairability. That's now a genuinely easy call.

So the real decision is cleaner than it looked:
1. **Cheapest:** GMKtec **direct at £2,660** — accept China-direct returns, save ~£800 vs Framework.
2. **Best value-for-trust:** **Framework ~£3,449** — only ~£350 over Amazon's GMKtec, Western brand, Linux-first. *This is probably your pick given everything you've said.*
3. Don't pay £3,100 for the GMKtec on Amazon — at that price just add £350 and get the Framework.

The linchpin is the Framework UK price, which I keep failing to pin. Let me try to nail it properly right now.

• WebSearch Framework Desktop 128GB UK price GBP "£" Ryzen AI Max+ 395 configurator 2026 4.9s 2.4 KB

• WebFetch https://frame.work/gb/en/products/desktop-diy-amd-aimax300/configuration/new 3.6s 399 B

Confirmed — Framework UK 128GB is **£3,449 barebones**, and crucially that's with **no storage**. Add a 1TB drive (~£265; Linux is free so skip the £120 Windows) and you're at **~£3,714 complete**. The US-vs-UK gap is brutal: $1,999 there, £3,449 here — a huge UK markup.

That **walks back my "only £350 more" claim** — that was barebones-vs-Amazon. Apples-to-apples (both ready to run, with storage), here's the real, now-verified picture:

| Box | Complete price | Storage | Brand / protection |
|---|---|---|---|
| **GMKtec direct** | **£2,659.99** | 2TB | Chinese, China-direct returns |
| **GMKtec Amazon** | **£3,099.96** | 2TB | Chinese, **UK buyer protection** |
| **Framework** | **~£3,714** | 1TB | US, Linux-first — but +£1,050 vs GMKtec direct, *less* storage, over your £3.5k line |
| **HP Z2 Mini G1a** | ~£3,500–4,500 | 1TB | Enterprise 3yr warranty, priciest |

All identical performance — same chip, same 256 GB/s, same tok/s.

So the honest landing:
- **Framework's UK markup kills its case.** It's ~£600 more than the Amazon GMKtec *and* ~£1,050 more than GMKtec direct, with half the SSD. You're paying ~£1k for the badge, not for speed or storage.
- **Your instinct was right.** The **Amazon GMKtec at £3,099.96** is the pragmatic sweet spot: UK consumer-rights returns as your safety net against the "too Chinese" worry, comfortably under £3.5k, 2TB included, same box. The £440 over direct *is* the insurance — and it's far cheaper insurance than the £600–1,000 Framework/HP premium for the same machine.
- **GMKtec direct at £2,660** only if you're comfortable with China-direct returns and want to save the £440.

My pick for you, given everything you've said: **GMKtec 128GB via Amazon UK, £3,099.96.** It resolves the whole thread — Linux, all-in-one, 128GB, big models, under budget, and Western buyer protection without the Western-brand tax.

Two things I should do now: my `index.html` has a wrong GMKtec figure (£1,620 = the 64GB config) and a wrong HP number. Want me to update the catalogue with these verified prices — GMKtec 128GB direct/Amazon, Framework £3,449+, HP street — and add a short "trustworthy-brand tax" note so future-you doesn't re-trip on this? I'll also log the decision in `memory.md`.

• WebFetch https://www.gmktec.uk/pages/return-and-refund-policy 5.5s 1.1 KB

Good find — and that genuinely strengthens the case for buying direct. Let me read it straight, the good and the sharp edges:

**What's reassuring:**
- **1-year limited warranty** (materials/workmanship) — and it applies *either way*, because it's GMKtec's manufacturer warranty. So it's **not** a reason to pay the Amazon premium; you get it on both.
- They have a **UK warehouse (Thetford, Norfolk)** and trade as "GMKtec UK" on a `.uk` domain. That matters more than it looks: it means the **UK Consumer Rights Act 2015 applies** regardless of the stated warranty. A "1-year limited warranty" can't override your statutory rights — goods must be satisfactory quality and last a *reasonable* time (claims possible well beyond a year). So direct isn't naked grey-import; you have UK law behind you.

**The sharp edges (this is what the Amazon £440 actually buys):**
- **14-day** no-reason return window only (Amazon gives 30, and far more leniently).
- **15% restocking fee** on opened/used units. On a £2,660 box that's a **~£399 hit** if you open it and decide it's not for you. Amazon = free return within 30 days, no fee.
- Returns go to an **address emailed after review** that "varies" — i.e. it *could* be China for some cases, at your hassle/cost initially. Amazon = click, print label, courier collects.
- For disputes you're chasing **Shenzhen GMK Technology Co.**; on Amazon you're dealing with Amazon UK and the A-to-z guarantee.

**So the £440 premium is no longer "the only protection vs none."** Both routes give you the 1-year warranty *and* CRA rights. The premium now only buys **return friction reduction + buying out the 15% restocking risk + an easier DOA/dispute path.**

That reframes it as a real judgement call:
- **Confident you'll keep it and it won't arrive dead?** → **GMKtec direct, £2,659.99.** The UK warehouse + 1yr warranty + CRA is legitimately enough. Save the £440.
- **Want frictionless returns and zero restocking-fee risk?** → **Amazon, £3,099.96.** The £440 is convenience/DOA insurance, not fundamental protection.

Honestly, given there's a real UK warranty and UK warehouse behind the direct option, I'd lean **direct at £2,660** now unless the no-quibble Amazon returns are worth £440 of peace of mind to you specifically. Either way you're well under £3.5k and it's the same machine.

Want me to lock all this into `index.html` — the verified GMKtec 128GB prices (direct £2,659.99 / Amazon £3,099.96), the warranty terms, the Framework £3,449+ and HP figures, plus a short "trustworthy-brand tax" note — and log the decision in `memory.md`?

Let me read the relevant sections of `index.html` so I edit them accurately.

• Read /home/megaproxy/claude/research/llm-box-uk/index.html 14ms 2.7 KB

• Read /home/megaproxy/claude/research/llm-box-uk/index.html 6ms 5.5 KB

• Read /home/megaproxy/claude/research/llm-box-uk/index.html 14ms 1.2 KB

Now let me read the unified-section intro and the verdict rows.

• Read /home/megaproxy/claude/research/llm-box-uk/index.html 13ms 3.5 KB

Let me read the hero tag/banner block so I can stamp the re-price date.

• Read /home/megaproxy/claude/research/llm-box-uk/index.html 17ms 1.5 KB

Now I'll make all the edits.

• Edit /home/megaproxy/claude/research/llm-box-uk/index.html 51ms 161 B

• Edit /home/megaproxy/claude/research/llm-box-uk/index.html 31ms 161 B

• Edit /home/megaproxy/claude/research/llm-box-uk/index.html 30ms 161 B

• Edit /home/megaproxy/claude/research/llm-box-uk/index.html 61ms 161 B

• Edit /home/megaproxy/claude/research/llm-box-uk/index.html 30ms 161 B

• Edit /home/megaproxy/claude/research/llm-box-uk/index.html 54ms 161 B

• Edit /home/megaproxy/claude/research/llm-box-uk/index.html 25ms 161 B

Now updating `memory.md` with the correction and the decision.

• Edit /home/megaproxy/claude/research/llm-box-uk/memory.md 45ms 160 B

Now let me commit and push.

• Bash error git add -A && git commit -q -m "Correct Build L pricing (128GB GMKtec £2,660 direct/£3,100 Amazon) + add brand-tax note … 2.3s 527 B

Commit landed locally but the remote has commits I don't have. Let me rebase onto it and push.

• Bash git pull --rebase origin main 2>&1 | tail -5 && echo "---" && git push -q origin main && echo "PUSHED" 4.6s 219 B

Locked in and pushed. (Heads up: the remote had one commit I didn't have locally — `86bdd2d` — so I rebased my change on top of it; the rebase was clean, no conflicts. If you pushed that from another machine, nothing's lost.)

Here's what changed in `index.html`:

- **Build L re-priced** — header now `~£2,660 direct total`; the BOM shows both lines: GMKtec **£2,659.99 direct** and **£3,099.96 Amazon UK (buyer protection)**. The `[00]` index table row and verdict row updated to match.
- **Brand-tax note added** to Build L's risk box and verdict: same chip / identical speed across GMKtec, Framework (£3,449 barebones, ~£3,710 w/ 1TB), and HP Z2 Mini G1a (~£3.5–4.5k street) — Western brands cost more for zero extra performance.
- **Warranty reality captured** — GMKtec UK's 1-yr warranty + Thetford warehouse means UK Consumer Rights Act applies either way; the £440 Amazon premium is framed as returns convenience + restocking-fee buyout, not fundamental protection.
- **£/GB corrected** — Strix Halo £12.7 → **£20.8/GB**, with a note that the old figure came from pricing the 64GB config.
- **Date stamps** — new `L RE-PRICED 2026-06-17` tag and a correction sentence in the re-priced banner.

`memory.md` got a full `2026-06-17` session-log entry: your budget/constraints (£6k 0% card, ~£3–3.5k target, Linux, all-in-one 128GB, brand-wary), all the verified prices, the warranty findings, the landing recommendation, and a note-to-self to stop trusting search snippets for prices.

So the catalogue now reflects the real decision: **GMKtec 128GB — £2,660 direct if you're confident, £3,100 via Amazon for the returns safety net** — both well under your ceiling, same machine the Western brands sell for ~£1k more.

One thing I left untouched: I didn't re-verify the other builds (A–J, K, M) this pass, so those prices still carry their 2026-06-05/06-07 dates. Want me to sync the wiki Dashboard + this folder's page to reflect the re-price, or leave that until you've actually decided direct-vs-Amazon?

Honest answer: **it depends entirely on what kind of "large" model**, and this is the single most important thing to understand before you buy — because the chip has one specific weakness.

The bottleneck is **memory bandwidth: 256 GB/s.** For text *generation*, speed ≈ bandwidth ÷ how much of the model gets touched per token. That splits large models into two very different camps:

| Model | Type | Realistic speed | Feel |
|---|---|---|---|
| 8–14B (Llama/Qwen) | dense | ~20–40 tok/s | Snappy |
| 32B Q4 | dense | ~9–13 tok/s | Comfortable |
| **70B Q4** | **dense** | **~4–8 tok/s** | **Usable but you feel it** |
| 120B+ | dense | ~2–3 tok/s | Painful |
| gpt-oss-120B | **MoE** | ~25–40 tok/s | Fast |
| **Qwen3-235B-A22B** | **MoE** | **~11 tok/s** | Comfortable |

**The key insight:** a *huge* MoE model (235B) runs **faster** than a *dense* 70B on this box, because MoE only activates ~22B of its parameters per token. So "large" in the MoE sense is the chip's sweet spot; "large" in the dense-70B-and-up sense is where it drags.

For reference, human reading is ~7–10 tok/s. So:
- **MoE big models (235B, gpt-oss-120B): no, not slow** — at/above reading pace. This is what the box is *for*.
- **Dense 70B: yes, somewhat slow** — fine for a chat where you read as it types, but long outputs (code, essays) you'll be waiting on.
- **Dense 100B+: too slow to enjoy.**

**The catch most people miss — time-to-first-token.** Those numbers are *generation* speed. *Prefill* (digesting your prompt/context before it starts replying) runs on the iGPU and is much slower than a 3090. So if you paste a long document or have a big chat history, expect a **noticeable pause (10–20s+) before the first word appears**, especially on the bigger models. Short prompts = near-instant; long-context/RAG/coding workloads = this is the thing that'll annoy you.

**Bottom line for you:**
- If your "large model" plan is **MoE** (Qwen3-235B, DeepSeek, gpt-oss-120B) → you'll be happy. Genuinely good for the money.
- If it's **dense 70B**, it's *usable* but not fast, and no Strix Halo box (GMKtec/Bosgame/Framework/HP — same chip) will be faster. The only ways to make dense-70B *fast* are a Mac Ultra (800 GB/s, ~3× quicker, but no Linux) or 2× RTX 3090 (but that's the loud multi-GPU rig, not an all-in-one).

So — what's the actual model you most want to run? If you name it, I'll give you the real expected tok/s and time-to-first-token for *that* one, so there are no surprises after you've spent the money.

Good — with the GMKtec's 256 GB/s in mind, the rule is simple: **anything ≤32B dense, or any MoE model, runs at comfortable-to-fast speeds.** Dense 70B+ is the only "large" category that drags. Here's a use-case-mapped shortlist of models that'll feel responsive on your box (reading pace is ~7–10 tok/s, so anything above that feels real-time):

| Use case | Model | Type / size (Q4) | Expected speed | Why it fits |
|---|---|---|---|---|
| **Coding (daily)** | **Qwen3-Coder-30B-A3B** | MoE, ~18GB | **~25–35 t/s** ⚡ | Only 3B active → fast *and* big-model-smart. Best all-round coder for this box. |
| Coding (max quality) | Qwen2.5-Coder-32B | dense, ~19GB | ~9–13 t/s | Strongest open coder at 32B; comfortable, not fast. |
| Coding/agentic (big) | **gpt-oss-120B** | MoE, ~63GB | **~25–40 t/s** ⚡ | ~5B active → flies despite size. Excellent tool-calling. Frontier-ish quality. |
| **Vision** | Qwen2.5-VL-7B | dense, ~6GB | ~30–40 t/s ⚡ | Fast, genuinely good OCR/screenshots/UI. |
| Vision (better) | Gemma 3 27B *(multimodal)* | dense, ~16GB | ~9–12 t/s | Strong image+text reasoning; one model for chat *and* vision. |
| Vision (mid) | Qwen2.5-VL-32B | dense, ~19GB | ~9–12 t/s | Best vision quality that still runs at a usable pace. |
| **NPCs / characters** | **Qwen3-30B-A3B** | MoE, ~18GB | **~25–35 t/s** ⚡ | Fast enough to serve many personas; one model, many system prompts. |
| NPCs (light/fast) | Gemma 3 12B / Mistral-Nemo 12B | dense, ~7–8GB | ~18–28 t/s ⚡ | Snappy; great for lots of short character turns. RP-finetunes exist on these bases. |
| NPCs (richer) | Mistral Small 24B | dense, ~14GB | ~12–16 t/s | Better writing/persona depth, still responsive. |
| **Agentic (openclaw etc.)** | gpt-oss-120B | MoE, ~63GB | ~25–40 t/s ⚡ | Best speed × tool-use combo — the agentic pick. |
| Agentic (purpose-built) | **Devstral Small 24B** | dense, ~14GB | ~12–16 t/s | Mistral's model *designed* for agentic coding (OpenHands/Cline/Aider). |
| Agentic (smartest) | Qwen3-235B-A22B | MoE, ~100GB @ Q3 | ~11 t/s | Strongest reasoning; fills the box alone, but comfortable speed. |

**A sensible "keep installed" loadout** (you swap on demand — they don't load at once):
- **gpt-oss-120B** — your heavy coding + agentic brain (~63GB).
- **Qwen3-Coder-30B-A3B** — fast everyday coding (~18GB).
- **Qwen2.5-VL-7B** (or Gemma 3 27B) — vision.
- **Qwen3-30B-A3B** or a 12B — NPCs/characters.

With 128GB you can hold the big MoE *or* 3–4 of the mid models (~14–19GB each) resident simultaneously — handy for running, say, a coder + a vision model + a character model at the same time. Tools like Ollama/llama.cpp swap them on demand from disk in seconds.

**Two honest caveats for your specific use cases:**
1. **Agentic + vision both stress the weak spot (prefill / time-to-first-token).** Agents send long contexts and many calls; images are compute-heavy to ingest. So you'll see a **pause before the first token** (a few seconds, longer with big context/images), even though generation itself is fast. MoE models (gpt-oss-120B, the -A3B Qwens) minimise this — another reason they're the right picks for agentic work. Keep agent context lean and it stays pleasant.
2. **"Many NPCs at once" = concurrency, not just speed.** The iGPU batches far worse than a real GPU, so 1 character replying = fast, but 10 characters replying *simultaneously* will queue and slow down. The fix is a fast small/MoE model + serving them sequentially or in small batches.

On **"openclaw"** — I'm not certain that's the exact name (closest I know are OpenHands, Cline, Aider, opencode, or a Claude-Code-style agent). The model advice above holds for any of them since they all need solid tool-calling + speed — gpt-oss-120B and Devstral are your best bets. If you tell me which one you mean, I'll confirm it runs well and suggest the right model + context settings for it.

Want me to add this "what to actually run" model table to Build L in `index.html` so it lives with the build?

Let me verify current image-gen performance on Strix Halo and what the current best models are, rather than guess.

• WebSearch Ryzen AI Max+ 395 Strix Halo Radeon 8060S Stable Diffusion SDXL Flux performance it/s ComfyUI ROCm 2026 11.2s 2.8 KB

• WebSearch best open image generation models 2026 Flux SD3.5 Qwen-Image Sana comparison current 14.0s 3.3 KB

Image gen works on this box but it's a different story from LLMs — let me get the structure right. Let me check the existing table markup so my additions match the styling.

• Bash grep -n "tablewrap\|<thead\|<th>\|\.tablewrap th\|table th\|class=\"decision\"\|\.decision" index.html | head -30 34ms 1.4 KB

• Read /home/megaproxy/claude/research/llm-box-uk/index.html 9ms 1.2 KB

Now I'll add both panels (the model loadout + the image-gen analysis) to the Build L area, matching the existing table styling.

• Edit /home/megaproxy/claude/research/llm-box-uk/index.html 46ms 161 B

Now a quick banner note and a memory.md log entry, then commit.

• Edit /home/megaproxy/claude/research/llm-box-uk/index.html 32ms 161 B

• Edit /home/megaproxy/claude/research/llm-box-uk/memory.md 36ms 160 B

• Bash git add -A && git commit -q -m "Add Build L model-loadout table + image-generation analysis Per-use-case fast-response m… 7.5s 87 B

Done — both panels are in `index.html` and pushed. Here's the answer to your image-gen question (also now captured in the doc):

## Image generation on the GMKtec — the honest picture

**It's a completely different bottleneck from LLMs.** Text generation is memory-*bandwidth*-bound, which is why your 128GB shines. Image generation is **compute-bound** — it's heavy matrix math on the GPU — so the limiter is the **Radeon 8060S iGPU** (40 RDNA 3.5 cores, roughly RTX 4060-class), *not* the memory. The 128GB barely helps here.

**Tooling is sorted now:** ROCm 7.x + ComfyUI officially support this chip (gfx1151) on Linux ([Phoronix tested it](https://www.phoronix.com/review/amd-rocm-7-strix-halo)), and AMD lists FLUX/SDXL as optimised targets. So it *works* — it's just not fast.

**Expected speed (1024px, estimates):**

| Model | Status | Time/image | 
|---|---|---|
| **Z-Image-Turbo** | newest, few-step | **~10–20s** ⚡ |
| **SDXL + Turbo/Lightning** | huge ecosystem | **~10–25s** ⚡ |
| SD 3.5 Large Turbo | open-weight | ~15–30s |
| FLUX.1 dev / FLUX.2 | quality king | ~1–2+ min |
| HunyuanImage-3.0 (80B) | fits only on 128GB | several min |

**What's good and current (2026):** [FLUX.2 / FLUX.1 dev](https://www.bentoml.com/blog/a-guide-to-open-source-image-generation-models) is the quality king; **SD 3.5** is the self-hosting favourite; **Z-Image-Turbo** is the new efficiency star (FLUX.2-level quality in a few steps — ideal for your box); **SDXL** still wins for sheer LoRA/finetune variety; and **HunyuanImage-3.0** (80B MoE) is the one thing where your 128GB is a genuine flex — no 24GB card can even load it.

**Bottom line:** for *occasional* image gen alongside the LLM/coding/NPC work, the GMKtec is perfectly usable — stick to the few-step "Turbo" models and you'll get images in ~10–30s. But if image generation were a *primary, high-volume* goal, a **used RTX 3090 is ~5–10× faster** for diffusion (compute is king here) — it just caps at 24GB. So: fine as a sometimes-thing, not the tool to pick if you're churning out images all day.

That tracks with everything else about this box: **brilliant capacity, modest compute/bandwidth.** It's a big-model *inference* machine first; image gen and dense-70B speed are the two areas where you feel the iGPU's limits.

Want me to also sync the wiki Dashboard/folder page to reflect today's re-price + the new Build L panels, or leave the wiki until you've locked the purchase?

That's a game-changer for this question — your **4070 is dramatically better for image generation than the GMKtec**, and it reframes the whole setup.

**Why the 4070 wins for diffusion:** image gen is compute-bound, and the 4070 brings two things the Strix Halo can't:
1. **CUDA** — the native, best-supported path for *every* diffusion tool (ComfyUI, Forge, A1111). No ROCm setup faff, everything "just works," every optimisation lands here first.
2. **Far more usable compute** — Ada Lovelace tensor cores vs a mobile-class iGPU.

**Real-world 4070 (12GB) speeds:**

| Model | 4070 | GMKtec (for contrast) |
|---|---|---|
| SDXL 1024px, ~30 steps | **~4–6s/image** ⚡ | ~10–25s |
| SDXL Turbo/Lightning (few steps) | **~1–2s** ⚡⚡ | ~10s |
| FLUX.1 dev (fp8/GGUF) | ~15–30s | ~1–2min |
| FLUX schnell (4 steps) | ~5–10s | ~30–60s |

So roughly **3–5× faster than the Strix Halo on SDXL**, with a much smoother software experience.

**The 4070's only limit is its 12GB VRAM:**
- FLUX needs to be **quantized** (fp8 or GGUF Q4/Q5) to fit — totally normal, minor quality cost, the whole community does it.
- No room for the giant stuff: HunyuanImage-3.0 (80B), big batches, high-res video — that's where VRAM, not speed, runs out.

**The strategic takeaway — you don't have to pick one box for everything.** If the 4070 lives in a desktop you're keeping, you've got a near-ideal complementary pair:

- **4070 desktop** → image generation (where it's genuinely good) + fast small-model LLMs (≤13B fit in 12GB).
- **GMKtec** → the big LLMs the 4070 *can't touch* — 70B, gpt-oss-120B, Qwen3-235B. That's literally the gap the 128GB box exists to fill.

That actually **strengthens the GMKtec purchase**: you're not buying it to do image gen (the 4070 already nails that) — you're buying it purely for big-model LLM inference, which is exactly its strength. The two machines cover each other's weak spots. The earlier "image gen is meh on Strix Halo" caveat basically stops mattering for you, because you'd never run diffusion on it when the 4070 is right there.

Is the 4070 in a separate desktop you'll keep running? If so, that two-box split is the setup I'd aim for. And — base 4070, or a Super/Ti/Ti-Super? (The Ti-Super's 16GB would let FLUX/SD3.5 run unquantized, which is a nice bump.) Want me to add a short "pair it with a discrete GPU for image gen" note to Build L while I'm at it?

Easiest path to test the current best models on a 4070 today is **ComfyUI** — it gets the newest models first, ships ready-made workflow templates, and handles 12GB fine. You can be generating in ~30 minutes. Here's the plan:

## 1. Install ComfyUI
- **Windows:** grab the **ComfyUI Desktop** one-click installer from [comfy.org](https://www.comfy.org/) → install → done. It auto-detects the 4070/CUDA.
- **Linux:** either the same Desktop AppImage, or manual:
  ```bash
  git clone https://github.com/comfyanonymous/ComfyUI
  cd ComfyUI && python -m venv venv && source venv/bin/activate
  pip install torch --index-url https://download.pytorch.org/whl/cu124
  pip install -r requirements.txt
  python main.py
  ```
- **Don't want to fiddle?** [Pinokio](https://pinokio.computer/) or **SwarmUI** are one-click installers (either OS) that wrap ComfyUI with a friendlier UI — good if you'd rather not touch a terminal.

## 2. First quick win — SDXL (proves it works, ~4–6s/image)
Download one SDXL checkpoint into `ComfyUI/models/checkpoints/`:
- **Juggernaut XL** or **Illustrious** from [Civitai](https://civitai.com) (community favourites), or plain `stable-diffusion-xl-base-1.0` from Hugging Face (~6.5GB).
In ComfyUI: **Workflow → Browse Templates → "SDXL"**, hit Queue. Instant gratification, plus the biggest LoRA/finetune ecosystem.

## 3. The current quality king — FLUX.1 dev (fits 12GB, ~20–30s/image)
The painless route for 12GB is the **all-in-one fp8 checkpoint** (model + text encoders + VAE in one file):
- Get **`flux1-dev-fp8.safetensors`** from the **Comfy-Org** repo on Hugging Face → drop in `models/checkpoints/`.
- Template: **"Flux"** → Queue. ~11GB, fits comfortably.
- Add **`flux1-schnell-fp8`** too — 4-step, ~5–10s/image, great for fast iteration before committing to a dev render.

## 4. Push the newest (optional)
- **SD 3.5 Large** — GGUF Q4/Q5 or fp8 fits 12GB (base file is gated on HF, needs a free login).
- **Z-Image-Turbo** — the new few-step star; check it has a ComfyUI workflow yet (it's very recent) — if so it'll be the fastest "good" option on your card.

## 4070-specific tips
- For FLUX/SD3.5 always use **fp8 or GGUF** quants — full-precision won't fit 12GB. (GGUF Q4/Q5 via "city96" quants if you want even more headroom + LoRA room.)
- Enable **xformers / sage-attention** for a speed bump (ComfyUI usually auto-uses it).
- If you ever hit out-of-memory, add `--lowvram` to the launch — but on a 4070 you rarely need it.
- **Civitai** is your hub for finetunes, LoRAs, and ready-made workflows you can drag-and-drop into Comfy.

**Suggested order to actually try:** SDXL (instant win) → FLUX schnell (fast FLUX taste) → FLUX dev (the "wow") → then SD3.5 / Z-Image if you want to compare.

What OS is the 4070 box on (Windows or Linux)? Tell me and I'll give you exact copy-paste install + the precise model download links/filenames for that setup so there's no guesswork.

Before I start downloading multi-GB models, let me check what's actually available in your environment — Docker, GPU passthrough to containers, and that the 4070 is visible.

• Bash echo "=== docker ===" ; docker version --format '{{.Server.Version}}' 2>&1 | head -3 ; echo "=== docker ps ===" ; docker… 7.1s 705 B

Docker's up, the 4070 is visible in WSL (12GB, driver 595.97), and you've got 935GB free. Now the critical check — GPU passthrough into containers:

• Bash echo "=== GPU in container test ===" ; timeout 180 docker run --rm --gpus all nvidia/cuda:12.4.0-base-ubuntu22.04 nvidia… 12.3s 306 B

GPU passthrough into containers works (`--gpus all` → RTX 4070), and Compose v5.1.4 is available. I'll set up ComfyUI + FLUX in Docker with the 4070 wired in. Let me scaffold it.

• Bash mkdir -p /home/megaproxy/comfyui-docker/data/{checkpoints,output,input,loras,vae} && cd /home/megaproxy/comfyui-docker &… 2.5s 269 B

Now let me write the Dockerfile and Compose config.

• Write /home/megaproxy/comfyui-docker/Dockerfile 3.1s 139 B

• Write /home/megaproxy/comfyui-docker/docker-compose.yml 3.0s 147 B

• Write /home/megaproxy/comfyui-docker/README.md 2.2s 138 B

Let me verify the FLUX model URL before pulling ~12GB, and kick off the Docker build in the background.

• Bash cd /home/megaproxy/comfyui-docker && echo "=== build (background) starting ===" && docker compose build > /home/megaprox… 4.4s 1.2 KB

Valid and public (~17.2GB — it's the all-in-one checkpoint bundling the T5 text encoder, so a single file does everything). Disk is fine. Let me start the download in the background alongside the build.

• Bash cd /home/megaproxy/comfyui-docker/data/checkpoints && nohup curl -L --retry 5 --retry-delay 5 -C - \ -o flux1-schnell-fp… 8.1s 437 B

Build is pulling the PyTorch base layer (~10MB/s). Let me check the download actually started.

• Bash cat /home/megaproxy/comfyui-docker/download.log 2>&1 | tail -5 ; echo "=== file ===" ; ls -lh /home/megaproxy/comfyui-do… 2.9s 2.0 KB

Both are flying — download at ~75 MB/s (~16GB, ETA ~3 min) and the build is pulling the PyTorch base. Let me start a background watcher that notifies me when both finish.

• Bash cd /home/megaproxy/comfyui-docker while pgrep -x curl >/dev/null 2>&1; do sleep 10; done echo "=== DOWNLOAD DONE ==="; l… 2.5s 291 B

Setup's underway. Here's where things stand:

- ✅ **Docker + GPU verified** — the 4070 (12GB, driver 595.97) passes through to containers.
- ✅ **ComfyUI stack scaffolded** at `~/comfyui-docker/` — Dockerfile (PyTorch CUDA 12.4 base + ComfyUI), `docker-compose.yml` with the GPU wired in and model/output folders bind-mounted to the host, plus a README.
- ⏳ **Building the image** (pulling PyTorch base, then installing ComfyUI deps).
- ⏳ **Downloading FLUX.1 [schnell] fp8** (~16GB, all-in-one checkpoint) at ~75 MB/s.

I started with **FLUX schnell** deliberately: it's Apache-2.0 (no Hugging Face login/license gate), and it's the fast 4-step variant — ideal for the 4070. FLUX dev is higher quality but gated; I can add it after if you accept its license and give me a token.

I've got a watcher running that'll ping me the moment both the build and download finish — then I'll start the container, confirm the 4070 is being used inside it, run a test generation to prove FLUX actually works end-to-end, and hand you the UI at **http://localhost:8188**. Sit tight.

• Bash cd /home/megaproxy/comfyui-docker echo "=== file size ==="; ls -lh data/checkpoints/flux1-schnell-fp8.safetensors 2>&1 e… 10.3s 431 B

**Download's done** ✅ — the full ~16GB FLUX.1 schnell checkpoint pulled in 3m31s at ~77 MB/s, sitting at `data/checkpoints/flux1-schnell-fp8.safetensors`.

The **build is on the final stretch** — it's pip-installing ComfyUI's Python deps now (downloading `av`, etc.), which is the last step before the image is ready. My watcher will ping me the second it finishes, and then I'll bring the container up, confirm the 4070 is active inside it, and run a test generation. Almost there.

Both done. Let me bring up the container and verify the 4070 is active inside it.

• Bash cd /home/megaproxy/comfyui-docker && docker compose up -d 2>&1 | tail -5 echo "=== waiting for startup ===" for i in $(s… 14.1s 890 B

Container's up, and ComfyUI sees everything it needs:
- ✅ **GPU active inside the container:** `cuda:0 NVIDIA GeForce RTX 4070`, 12GB, PyTorch 2.5.1+cu124
- ✅ **FLUX checkpoint detected:** `flux1-schnell-fp8.safetensors`

Now the real proof — a test generation through the API. Let me write a minimal FLUX schnell workflow and run it.

• Write /home/megaproxy/comfyui-docker/test_flux.py 3.0s 141 B

• Bash cd /home/megaproxy/comfyui-docker && python3 test_flux.py 2>&1 echo "=== output files on host ==="; ls -lh data/output/ … 117.7s 291 B

First run includes loading the 16GB checkpoint (the all-in-one bundles T5, so it'll juggle VRAM + system RAM on the 12GB card), so it takes a bit. Let me peek at progress.

• Bash error sleep 45; echo "=== test output ==="; cat /tmp/claude-1000/-home-megaproxy-claude-research-llm-box-uk/f718d7f7-ddff-4972… 3.3s 510 B

• Bash echo "=== test output ==="; cat /tmp/claude-1000/-home-megaproxy-claude-research-llm-box-uk/f718d7f7-ddff-4972-bae8-1638… 6.6s 481 B

Right, foreground sleep is blocked. Let me just read the interim output directly.

It's loading the model now — the repeated "Pin error" warnings are ComfyUI offloading the 16GB checkpoint into system RAM because it's bigger than the 4070's 12GB (non-fatal, just means the first run is slower while it sets up). The Python script's "queued" line is buffered, so the empty output is expected until it finishes. Let me confirm it's actively processing.

• Bash echo "=== queue ==="; curl -s http://localhost:8188/queue 2>&1 | python3 -c "import sys,json; d=json.load(sys.stdin); pr… 8.5s 106 B

It's running — **GPU at 65% util, 11.8GB VRAM in use**, FLUX is actively denoising on the 4070. Queue shows 1 job running. The first generation is slow because it's loading the 16GB model (with offload to system RAM, since it's bigger than 12GB VRAM), but it's working. I'll get pinged when the test image lands and report the result + timing.

• Bash cat /tmp/claude-1000/-home-megaproxy-claude-research-llm-box-uk/f718d7f7-ddff-4972-bae8-163871eb0701/tasks/bzfykb1f7.out… 5.7s 126 B

Timed out at 300s with no image. Let me see what actually happened — check the job history for errors and the container logs.

• Bash echo "=== queue now ==="; curl -s http://localhost:8188/queue 2>&1 | python3 -c "import sys,json; d=json.load(sys.stdin)… 8.1s 165 B

The pipeline is correct (GPU's working, no errors) but it timed out: the 16GB all-in-one checkpoint is too big for the 4070's 12GB, so ComfyUI is streaming it from system RAM — and in WSL2 that triggers the "Pin error" flood and makes it painfully slow. That's the wrong setup for a 12GB card.

The proper 4070 recipe is **GGUF-quantized FLUX** that fits entirely in VRAM (no streaming). Let me switch to that. First, stop the stuck job and restructure the mounts.

• Bash cd /home/megaproxy/comfyui-docker && docker compose down 2>&1 | tail -2 mkdir -p data/{unet,clip,custom_nodes} echo "===… 18.4s 251 B

Now update the Compose mounts and Dockerfile, then kick off the GGUF model downloads.

• Edit /home/megaproxy/comfyui-docker/docker-compose.yml 3.8s 157 B

• Edit /home/megaproxy/comfyui-docker/Dockerfile 5.2s 149 B

Let me verify the GGUF model URLs before downloading, and rebuild the image (adds the `gguf` package) in the background meanwhile.

• Bash cd /home/megaproxy/comfyui-docker echo "=== rebuild (bg) ==="; docker compose build > rebuild.log 2>&1 & echo "build PID… 4.4s 280 B

Three are good (unet GGUF 6.8GB, t5xxl 4.9GB, clip_l 246MB). The Black Forest VAE is gated (401). Let me start the three confirmed downloads now and hunt for an ungated VAE mirror in parallel.

• Bash cd /home/megaproxy/comfyui-docker # start the 3 confirmed downloads in background nohup curl -sL --retry 5 -o data/unet/… 356ms 497 B

None of the VAE mirrors are ungated. No problem — the 16GB all-in-one checkpoint we already downloaded *contains* the VAE, so I'll extract it from there once the container's up. Let me check the rebuild and download progress.

• Bash cd /home/megaproxy/comfyui-docker echo "=== rebuild status ==="; tail -3 rebuild.log 2>&1 | tr '\r' '\n' | grep -vE '^\s… 2.2s 538 B

Rebuild's done and the image is built. clip_l finished; t5xxl (2.4/4.9G) and the GGUF unet (2.2/6.8G) are still pulling. Let me bring the container up now (so I can extract the VAE while downloads finish) and confirm the GGUF node loaded.

• Bash cd /home/megaproxy/comfyui-docker && docker compose up -d 2>&1 | tail -2 echo "=== wait for API ===" for i in $(seq 1 18… 13.6s 414 B

GGUF node loaded ✅, and the VAE keys are prefixed `vae.` — easy to extract. Let me pull the VAE out of the all-in-one checkpoint into a standalone file.

• Bash docker exec comfyui python3 -c " from safetensors import safe_open from safetensors.torch import save_file src='/app/mod… 3.6s 523 B

VAE extracted (320MB, 244 tensors) ✅. The GGUF unet (5.9/6.8G) and t5xxl (4.6/4.9G) are nearly done. Let me write the GGUF test workflow and start a watcher for the last downloads.

• Write /home/megaproxy/comfyui-docker/test_flux_gguf.py 7.5s 146 B

• Bash cd /home/megaproxy/comfyui-docker while pgrep -x curl >/dev/null 2>&1; do sleep 5; done echo "=== DOWNLOADS DONE ===" ls… 2.5s 291 B

Everything's staged. The watcher will ping me when the last two files finish (both ~90%+ done), then I'll run the GGUF test generation. Current state:

- ✅ Container up, GGUF custom node loaded, 4070 active
- ✅ VAE extracted from the all-in-one checkpoint (320MB standalone `ae.safetensors`)
- ✅ clip_l downloaded
- ⏳ GGUF unet (5.9/6.8GB) + t5xxl encoder (4.6/4.9GB) finishing

The GGUF Q4 unet (~6.8GB) + fp8 T5 (~4.9GB) + VAE all fit inside the 4070's 12GB **with no streaming**, which is what kills the "Pin error" slowness — this run should be fast. Standing by for the downloads.

• Bash cat /tmp/claude-1000/-home-megaproxy-claude-research-llm-box-uk/f718d7f7-ddff-4972-bae8-163871eb0701/tasks/buhzqaudi.out… 2.4s 615 B

• Bash cd /home/megaproxy/comfyui-docker && python3 test_flux_gguf.py 2>&1 3.2s 291 B

All files complete and match expected sizes exactly. Now the real test — GGUF FLUX generation on the 4070:

• Bash echo "=== interim ==="; cat /tmp/claude-1000/-home-megaproxy-claude-research-llm-box-uk/f718d7f7-ddff-4972-bae8-163871eb… 3.3s 598 B

This run is healthy — **GPU at 98%, model fully loaded into VRAM** (10.3GB used, fits the 12GB cleanly): the log says `loaded completely ... full load: True`, meaning no streaming this time. The GGUF unet loaded correctly (`gguf qtypes: F32/Q4_K/F16`). It's actively sampling now. The remaining "Pin error" lines are just cosmetic WSL2 warnings during the load phase, not the per-step slowdown we had before. Waiting on the finished image.

• Bash cat /tmp/claude-1000/-home-megaproxy-claude-research-llm-box-uk/f718d7f7-ddff-4972-bae8-163871eb0701/tasks/bpyznay15.out… 2.7s 212 B

**It works.** 🎉 First image generated in **18 seconds** (and that included the cold model load). Let me view it and run a second one to get the warm steady-state time.

• Bash cd /home/megaproxy/comfyui-docker && python3 test_flux_gguf.py 2>&1 | tail -1 ls -lh data/output/ 2>&1 | tail -3 11.8s 256 B

• Read /home/megaproxy/comfyui-docker/data/output/flux_gguf_00001_.png 155ms 153.9 KB

That's a proper FLUX image — a genuine, detailed bookshop interior at golden hour, exactly to prompt. And the numbers are great:

- **Cold (first) generation: 18s** (includes model load)
- **Warm generation: 9.3s** per 1024px image (FLUX schnell, 4 steps)
- **VRAM: ~10.3GB** — fits the 12GB cleanly, no streaming

Let me give you a simple CLI generator and fix up the docs to match the working GGUF setup.

• Write /home/megaproxy/comfyui-docker/gen.py 2.5s 135 B

• Write /home/megaproxy/comfyui-docker/README.md 3.5s 148 B

Let me confirm the CLI generator works end-to-end with a fresh prompt.

• Bash cd /home/megaproxy/comfyui-docker && python3 gen.py "a friendly robot reading a book in a sunny library, cinematic, high… 15.7s 130 B

• Read /home/megaproxy/comfyui-docker/data/output/flux_00001_.png 149ms 149.4 KB

Done — and it works on your 4070. Generated to-prompt in 13s. **FLUX is fully set up and verified.**

Let me note the real-world result in the research memory (it corroborates the catalogue's image-gen claims), then I'll give you the rundown.

• Edit /home/megaproxy/claude/research/llm-box-uk/memory.md 53ms 160 B

• Bash git add -A && git commit -q -m "memory: log 4070 FLUX real-world datapoint (~9s/img GGUF Q4 on 12GB)" && git pull --reba… 3.4s 41 B

**FLUX is set up, tested, and working on your 4070.** Here's everything:

## Results
| | |
|---|---|
| Warm generation | **~9s** per 1024px image |
| Cold (first) | ~18s incl. model load |
| VRAM used | ~10.3GB (fits 12GB, no streaming) |
| Model | FLUX.1 schnell, GGUF Q4 |

Both test images came out clean and exactly to prompt (the bookshop + the robot-in-a-library).

## How to use it

**UI:** open **http://localhost:8188**

**CLI (easiest):**
```bash
cd ~/comfyui-docker
python3 gen.py "a red dragon over a snowy mountain, cinematic"
python3 gen.py "portrait of a knight" --steps 6 --w 832 --h 1216 --seed 42
```
Images appear in `~/comfyui-docker/data/output/`.

**Container control:**
```bash
cd ~/comfyui-docker
docker compose up -d     # start
docker compose down      # stop
docker compose logs -f   # logs
```

## What I built
- `~/comfyui-docker/` — Dockerfile (PyTorch CUDA 12.4 + ComfyUI + GGUF node), `docker-compose.yml` (4070 wired in, host-mounted model/output folders), `README.md`, `gen.py`.
- **Models:** GGUF Q4 unet + fp8 T5 + clip_l + VAE — all sized to fit 12GB.

## One thing worth knowing
I started with the all-in-one fp8 checkpoint (16GB), but that's **too big for 12GB** — it streamed from system RAM, flooded WSL2 "Pin error" warnings, and was painfully slow (timed out). Switching to **GGUF Q4** (fits entirely in VRAM) is the correct recipe for a 12GB card — that's what got you to 9s/image. The README documents this and the upgrade paths.

## Easy next steps (all in the README)
- **Higher quality:** swap to `Q5_K_M`/`Q6_K` GGUF (slightly slower).
- **FLUX.1 dev** (better prompt-following, gated — needs an HF licence accept + token).
- **SDXL** for the huge LoRA/finetune ecosystem and ~2–6s images.

Want me to add FLUX dev or drop in an SDXL model while I'm here?

• Bash cd /home/megaproxy/comfyui-docker && python3 -c " from PIL import Image im=Image.open('data/output/flux_00001_.png') key… 2.5s 199 B

Let me check whether you can just drag a generated image back into the UI to get the workflow (depends on what metadata got embedded), so I give you the path that actually works.

You can use it two ways. The **CLI works right now** (no setup); for the **web UI** I'll install a ready-made workflow so you just open it and click. The generated PNGs only carry the API prompt (not a UI graph), so dragging them in isn't reliable — a proper workflow file is better. Let me create and install one.

• Write /home/megaproxy/comfyui-docker/data/user/default/workflows/flux-schnell-gguf.json 1.1s 179 B

Now mount the user dir so the workflow shows in the UI, validate the JSON, and recreate the container.

• Edit /home/megaproxy/comfyui-docker/docker-compose.yml 3.0s 157 B

• Bash cd /home/megaproxy/comfyui-docker echo "=== validate workflow JSON ==="; python3 -c "import json; json.load(open('data/u… 17.0s 436 B

Everything's installed and the container's back up with the workflow mounted. Here's how to use it:

## Option A — Web UI (to play around)
1. Open your browser (on Windows) to **http://localhost:8188**
2. In the **left sidebar**, click the **Workflows** icon (📁) → open **`flux-schnell-gguf`** — it loads the whole graph ready to go.
3. Type your prompt in the **top-left text box** (the bigger "CLIP Text Encode" node — the lower empty one is the negative, leave it blank for schnell).
4. Click the big **Run / Queue Prompt** button (bottom-center).
5. The image appears in the **Save Image** node on the right, and is also written to `~/comfyui-docker/data/output/`.
6. Each run auto-randomizes the seed, so hit Run again for variations.

Keep these as-is (they're tuned for schnell): **steps 4, cfg 1.0, sampler euler, scheduler simple**. Change resolution in the "Empty Latent" node (e.g. 832×1216 for portrait).

## Option B — Command line (fastest, no clicking)
```bash
cd ~/comfyui-docker
python3 gen.py "a red dragon over a snowy mountain at dusk, cinematic"
python3 gen.py "portrait of a cyberpunk knight" --w 832 --h 1216 --seed 42
```
Images land in `~/comfyui-docker/data/output/` (browsable from Windows at `\\wsl$\...` too).

## Starting / stopping
```bash
cd ~/comfyui-docker
docker compose up -d     # start (it's running now)
docker compose down      # stop when you're done
docker compose logs -f   # watch logs
```

**Speed reminder:** the *first* image after starting takes ~18s (loading the model into VRAM); every one after that is ~9s.

That's it — open localhost:8188, load the `flux-schnell-gguf` workflow, type, and Run. Want me to also drop in **SDXL** (2–6s images, massive LoRA/style ecosystem on Civitai) as a second workflow, or wire up **FLUX dev** for higher quality?

You asked in tokens/**minute** — here it is (just our tok/s figures ×60). These are *generation* speed on the GMKtec's Ryzen AI Max+ 395 (256 GB/s), projected from published Strix Halo benchmarks — not yet measured on an actual unit:

| Model | tok/sec | **tok/min** | A ~500-token reply takes |
|---|---|---|---|
| 8–14B dense | ~20–40 | **~1,200–2,400** | ~15–25s |
| **Qwen3-30B-A3B** (MoE) | ~25–35 | **~1,500–2,100** | ~15–20s |
| **gpt-oss-120B** (MoE) | ~25–40 | **~1,500–2,400** | ~13–20s |
| 32B dense (Q4) | ~9–13 | **~540–780** | ~40–55s |
| Qwen3-235B-A22B (MoE) | ~11 | **~660** | ~45s |
| **70B dense (Q4)** | ~4–8 | **~240–480** | ~1–2 min |

**The pattern, in tok/min terms:**
- **MoE models (30B-A3B, gpt-oss-120B): ~1,500–2,400 tok/min** — this is the box's happy place. Big, capable models at a brisk pace.
- **Dense 70B: ~240–480 tok/min** — the slow lane. A long reply is a 1–2 minute wait.

**Three honest caveats:**
1. These are **decode (generation)** numbers. They *don't* include prefill — the pause before the first token while it reads your prompt. On long context or agentic/RAG use, that pause can add several seconds (the iGPU's weak spot).
2. **Tokens/min drops as context grows** — a 200-token prompt runs at the rates above; at 16k+ context the KV cache slows decode noticeably.
3. Real numbers depend on quant level and backend (llama.cpp Vulkan vs ROCm). Treat these as ±20%.

If you mostly live in **MoE-land** (which your coding/agentic/NPC plan fits well), you'll see **~1,500–2,400 tok/min** day-to-day. If you insist on **dense 70B**, budget for **~300–400 tok/min** and the patience that implies — and remember no Strix Halo box (GMKtec, Bosgame, Framework, HP) does better, since it's the same silicon.

Want me to convert these into something more concrete for your use cases — e.g. how long a typical coding response, an agent step, or an NPC line would actually take?