MiTAC Proposes Two-Server AI Inference Architecture at SC26 – Unite.AI

0
2



MiTAC Computing unveiled a two-server AI inference architecture with Skymizer at SC26 in a November 17, 2026 announcement, proposing 2 TB of decode-side accelerator memory for a system designed to run models such as the 2.8-trillion-parameter Kimi K3 on premises.

The architecture is part of a wider portfolio spanning software orchestration, high-density liquid cooling, storage and an upcoming AMD rack. Some supporting systems are being demonstrated live at SC26, while others remain proposed or in development, making the announcement a mix of current demonstrations and future infrastructure designs.

Separate Servers for Prefill and Decode

The proposed prefill platform is the 8U G8825Z5, using AMD EPYC 9575F or 9755 processors and Instinct MI350X GPUs. It would transfer the resulting key-value cache to the decode server over high-speed networking. For decode, the proposed air-cooled 4U G4520G6 configuration combines Intel Xeon 6 processors with eight Skymizer HTX301-4S accelerator cards, each carrying 256 GB of memory.

Prefill is the compute-intensive stage, while decode is memory-intensive, so separating them is intended to let enterprises provision each for its own workload. MiTAC says the on-premises decode system is designed for complex, deep-reasoning agentic workloads. The architecture shifts part of the scaling problem from adding GPUs to balancing accelerator memory with the network connection between stages.

Live Cooling and Data Density at SC26

MiTAC’s live demonstrations address the physical demands around that inference design. A 52U liquid-cooled rack integrates 12 G4826Z5 servers and 96 AMD Instinct MI355X GPUs, a configuration the company says delivers 50% higher GPU density than conventional deployments.

A proof of concept with ZutaCore uses waterless two-phase direct liquid cooling on the G4826Z5, vaporizing dielectric fluid directly at the chip. MiTAC reports support for thermal design powers above 4,000W, heat flux above 250W/cm² and an annualized partial power usage effectiveness of 1.03. The demonstration targets the thermal limits created by concentrating high-power accelerators inside dense racks.

The storage demonstration combines DDN Infinia with seven GC68C-B8056 1U storage servers and is designed to supply continuous AI data streams without I/O bottlenecks. That addresses a different constraint from cooling: reducing the risk that accelerators wait for data instead of processing inference and agent workloads.

Software and the Upcoming Helion Rack

Canonical Ubuntu supplies the certified operating-system layer, Rafay adds dynamic GPU-as-a-Service orchestration across distributed workloads, and MiTAC’s MiCoreView software provides pod-level telemetry and unified data-center visibility. Together with Infinia, those components cover operating systems, data delivery, GPU allocation and monitoring within the same infrastructure stack.

MiTAC is also collaborating with AMD on an upcoming ORv3 44OU Helion Agentic AI Rack. MiTAC says it will scale to 32,768 AMD Zen 6 CPU cores for hyperscale and enterprise environments.

AMD’s Helion documentation describes an open rack-scale reference design built on Open Compute Project (OCP) standards, with CPU-only hyperscale configurations and enterprise versions that mix CPU and GPU blades. Its air-cooled inference version supports up to 16 Instinct MI350P GPUs, and customers can mix blade types to match different agent workloads. The enterprise configuration is intended to combine agent execution and inference in one rack, unlike Helios, which is built for frontier-scale training and large-scale inference with a unified scale-up GPU domain.

The practical implication is that deployable on-premises inference depends on the decode memory working in concert with the prefill network, storage throughput, rack cooling and orchestration; MiTAC’s proposal makes those connections part of the capacity calculation alongside the GPUs themselves.



Source link

LEAVE A REPLY

Please enter your comment!
Please enter your name here