A homelab builder has pushed a 2.8-trillion-parameter AI model through a CPU-only setup built around second-gen Optane persistent memory, according to a Reddit post. The system runs Moonshot AI’s Kimi K3 on Intel Optane PMem 200 modules paired with a 40-core 3rd-gen Xeon Scalable processor, with no GPU handling inference.
The result is functional but far from fast: roughly 0.85 tokens per second with 1-bit quantization. The project follows an earlier build by a different user that used first-generation Optane memory.
Why Optane Makes the Build Possible
Kimi K3 is not a model designed for home hardware. Moonshot AI points users toward data center inference engines, and even the smallest community quantization requires at least 610GB of RAM. With DRAM prices elevated, secondhand Optane offers a way to reach that capacity at what the builder describes as a fraction of the cost.
Intel introduced the Optane Persistent Memory 200 Series in 2020 for 3rd-gen Xeon Scalable CPUs, offering 128GB, 256GB, and 512GB modules. On two-socket Xeon platforms, the second generation runs at a higher effective MT/s rate than PMem 100 and can deliver up to 32% more memory bandwidth on average.
App Direct Mode and the Memory Trade-off
Optane can operate in Memory Mode, where it acts as system memory with DRAM serving as cache, or in App Direct Mode, where it appears as a separate persistent region that software maps directly. The first build used Memory Mode; this one uses App Direct Mode, which the earlier builder said can give more control over where model tensors live.
Optane is slower than DRAM, and the performance gap widens at higher quantization. At 3-bit quantization, the second build dropped to 0.75 tokens per second. One commenter noted that a 50,000-token task would take roughly 13 hours at about 1 token per second, while another pointed out that overnight batch jobs are less sensitive to speed.
For comparison, the first build reached about 4 tokens per second, but that setup used an RTX 3060 to handle part of the model alongside Memory Mode, 2-bit quantization, and the smaller 1-trillion-parameter K2.5 model.
Heat Becomes the Limiting Factor
The Optane DIMMs produced extreme heat under sustained inference, enough that the builder warns it could endanger neighboring DRAM. The advice is blunt: treat active DIMM cooling as mandatory. Dual 12W blower fans aimed at the modules brought temperatures under control.
Intel’s specifications call for forced airflow of 2.4 to 2.7 meters per second, a level typical of data centers rather than home systems. Each Optane module is rated at 12W or 15W regardless of capacity, so four sticks could place up to 60W of heat directly next to system memory. Throttling begins at 83 degrees Celsius, with a thermal trip shutdown at 85 degrees, compared with typical DDR4 server memory rated for 85 degrees and up to 95 degrees in an extended range.
Room Left for Tuning
The reported performance figures are pre-tuning, meaning thread counts and NUMA affinity have not been optimized, so there is room for improvement. A commenter also raised the idea of using Memory Mode with DRAM caching for the most-used experts.
The builder plans to add GPUs and load parts of the model into DRAM as a faster tier alongside the Optane, with more benchmarks expected.
