Clustering local boxes, and two ways to share them
The models people want to run locally are bigger than one consumer box. Kai's Every Way to Get 512GB of VRAM for Local AI (October 17, 2026) shows that most realistic 512 GB-class builds are clusters: two Mac Studios, four DGX Sparks, two Strix Halo mini PCs, two MI50 machines over 10 GbE. How those boxes connect affects both speed and how well they can be shared.
Macs: Thunderbolt 5 with RDMA
Apple added RDMA over Thunderbolt in macOS 26.2 (Apple TN3205). According to the exo README, it works on Macs with Thunderbolt 5, is enabled once from Recovery with rdma_ctl enable, and lets exo split models across machines. The README cites tensor-parallel speedups of up to 1.8× on two devices and 3.2× on four.
Practical limits, from Jeff Geerling's hands-on test and the exo docs:
- Real-world Thunderbolt throughput of about 50–60 Gbps, with much lower latency than Ethernet.
- No Thunderbolt 5 switches. Every Mac connects directly to every other Mac, which in practice means clusters of about four.
- All nodes must run the exact same macOS version. Geerling also reported some early stability issues.
- Mac Studio also has 10 Gb Ethernet built in.
This is why the video's orderable Mac "512" is two 256 GB M5 Ultras linked over Thunderbolt 5. MacRumors reports that Apple says four clustered Mac Studios can run inference up to 3× faster than one.
Strix Halo and similar boxes: 10 GbE and a PCIe slot
AMD Strix Halo mini PCs usually ship with 10 GbE and often an internal PCIe slot. The video's example, the Minisforum MS-S1 MAX (official specs), has:
- 2× 10GbE (Realtek RTL8127)
- 2× USB4 v2 (80 Gbps) and 2× USB4 (40 Gbps)
- One full-length x16 slot, wired as PCIe 4.0 x4. ServeTheHome confirms the slot is physically x16 but electrically x4, and takes half-height cards.
So yes, you can in theory put a single- or dual-port 100 GbE NIC in that slot. But a PCIe 4.0 x4 link carries about 64 Gbps raw (our estimate: 4 lanes × 16 GT/s, slightly less after encoding and protocol overhead). The port would link at 100 Gb, but throughput would be capped below that, and a second port would add redundancy rather than bandwidth. That is still about 5–6× the built-in 10 GbE (estimate). Other Strix Halo boards wire their slots differently, so check the lane count for your exact model before buying a NIC. We have not tested this setup.
Cheap 100 GbE switching exists
You don't need a datacenter switch to connect four boxes. The MikroTik CRS504-4XQ-IN has four 100 GbE QSFP28 ports (each can break out to 4×25 GbE), draws about 25 W, and has a $799 suggested price. Retailers list it lower, for example $638.99 at Link Technologies at the time of writing. Compare the video's report that partner versions of NVIDIA's own switch for Sparks were selling for $7,000–$9,000. Add NICs, optics or DAC cables to your budget.
The video's MI50 build shows the low end: 16 cards across two machines over plain 10 GbE, running GLM-5.2 at 12.2 tok/s. It works, but slowly.
Two ways to share a box (or cluster)
Once the hardware exists, there are two broad ways to share it.
1. Lease a share of the day
The renter gets the whole box, or a whole cluster, for a block of time. A host might keep 9-to-5 for themselves and lease out nights.
- Pros: strong isolation, predictable performance, and the renter chooses the model and context. Good for fine-tuning, long-context jobs, and work that needs every byte of memory (GLM-5.3 at long context, for example).
- Cons: batching only happens within the renter's own work. One user at 87.7 tok/s leaves the ~3.9× aggregate headroom the four-Spark numbers show unused. Loading a new model for each lease takes time, and a 300–500 GB model can take minutes to load.
- On Hypermesh today: time-based leases are live, with the failure-refund rules on our pricing page. Day-ahead booking is listed as coming soon.
2. Share one well-sized model (or a set of models)
The host keeps a model loaded and serves many renters at once. Requests are batched, and each renter pays per token.
- Pros: batching efficiency, which is where the 342.7 vs. 87.7 tok/s gap comes from, and no model load per renter. This is how the video's 16-session payback math works.
- Cons: per-user speed drops (≈21 tok/s at 16 users on four Sparks, by our estimate). Long prompts from one user slow everyone else's decode. KV cache has to be budgeted per user.
- KV budgeting decides which model to share. A model with a small cache, like DeepSeek V4.1 Flash at 890 bytes/token, can give each user a long context. A model with a large cache, like GLM-5.3 at ~88 KB/token, can serve many users only with short contexts. A host with 512 GB might do better serving a smaller model such as GLM-5.3 Flash (which the video suggests) to many users than a giant model to a few.
- A set of models: a cluster can split its memory, for example one large model for hard tasks and a small fast one for routine calls, so each request goes to the cheapest model that can handle it. On Hypermesh, per-token access to posted models is live today. Automatic model selection is planned, not available yet.
Choosing
| If your work is… | Prefer |
|---|---|
| Private, long-context, or needs custom weights | Time-based lease |
| Many short agent calls, latency-tolerant | Shared model, per token |
| Interactive coding where you wait on every reply | Lease, or a shared model kept at low concurrency |
| Spiky and unpredictable | Shared model |
Try it
Hypermesh offers time-based leases and per-token model access on enrolled, checked hardware, at posted prices for each hardware class and model. Renters pay in USDC, and hosts are paid through Stripe Connect. Whether you are wiring up four Strix Halo boxes with a 100 GbE switch or have one Mac Studio with idle nights, see what fits at hyperme.sh.
Sources: Kai, "Every Way to Get 512GB of VRAM for Local AI" (Oct 17, 2026); Apple TN3205; exo README; Jeff Geerling; MacRumors; Minisforum MS-S1 MAX specs; ServeTheHome review; MikroTik CRS504-4XQ-IN. PCIe bandwidth and per-user figures are our estimates.