Meta has introduced MTIA 300 as the first member of its in-house MTIA family designed to accelerate training and inference for ranking and recommendation models. The company says MTIA 300's built-in NICs and dedicated communication-offloading engines address the unique communication patterns of training these models, claiming performance advantages over general-purpose GPUs.
INTEGRATED NETWORKING AND NIC CHIPLETS INSIDE THE CHIP
MTIA 300 integrates networking directly within the chip package. Two network chiplets, each with six custom 800 Gbps RDMA NICs, provide 1.2 TB/s of total I/O bandwidth without crossing a PCIe bus, removing the host-device–NIC bottleneck common in traditional GPU architectures. TechStaged has also covered Google Details AAOS SDV: Secure-by-Design Android Automotive OS Renews Vehicle Software Trust with Virtualization, Hardware Identity, and Memory Safety.
- Two network chiplets with 12 NICs total
- 1.2 TB/s total I/O bandwidth without PCIe crossing
- NICs configured for scale-up within a rack and scale-out across racks
OFFLOADING COMMUNICATION FROM THE COMPUTE GRID
The design includes 16 dedicated message engines (MEs) that handle all communication. Each ME contains an NIC interface and a near-memory compute (NMC) block that performs reductions at 128 bytes per cycle, enabling line-rate execution of collectives like AllReduce and ReduceScatter without using compute resources.
- 16 MEs for autonomous communication
- NMC blocks enable reductions at 128 bytes/cycle
- Express doorbells reduce latency by eliminating extra memory reads
CO-DESIGNED SOFTWARE FOR COMMUNICATION: HCCL
Meta’s co-designed communication library, HCCL, compiles each collective into subgraphs dispatched to the MEs for autonomous execution. After instructions are copied into high-bandwidth memory, the CPU is not involved in the ongoing communication. HCCL integrates with PyTorch interfaces and supports topology-aware algorithms to minimize cross-rack traffic.
- HCCL translates collectives into self-contained subgraphs
- CPU becomes uninvolved after instruction submission
- PyTorch integration with torch.compile
PERFORMANCE AND EFFICIENCY ATTRIBUTES
MTIA 300 quotes up to 940 GB/s of communication bandwidth within a single rack. In a production-recommendation model with about 150 billion parameters running across 40 accelerators, MTIA 300’s total communication time is described as 3.9 times faster than the equivalent GPU cluster.
- Up to 940 GB/s intra-rack bandwidth
- 3.9× faster communication vs. GPU cluster on a large model
HARDWARE FOOTPRINT AND MODEL-PARALLEL CONSIDERATIONS
The chip includes 216 GB of HBM3E memory and supports a 1:1 CPU-to-accelerator ratio, enabling CPU offloading for numerically intensive optimizer tasks. Embedding tables can account for over 99% of a model’s parameters, motivating hybrid parallelism with frequent AllReduce, AllToAll, and AllGather operations.
- 216 GB HBM3E memory
- 1:1 CPU-to-accelerator ratio
- Embedding tables containing >99% of parameters
BROADER IMPLICATIONS AND NEXT STEPS
Meta frames MTIA 300 as more than a single chip; the architectural approach—integrated networking, offloaded collectives, and compute-communication co-design—serves as a foundation for future AI silicon. Meta notes related research papers, including ISCA ’26 work and a forthcoming SC26 publication, to expand on HCCL and collective offloading strategies.
- Network-as-a-first-class constraint
- Foundation for next-generation AI silicon
- References to ISCA ’26 and SC26 papers
RELATED COVERAGE
- Google Details AAOS SDV: Secure-by-Design Android Automotive OS Renews Vehicle Software Trust with Virtualization, Hardware Identity, and Memory Safety
- AudioWorldSim: Realistic Binaural Audio Datasets For World Models Debuts on arXiv
- Time-Aware Transformer-Based Prediction Model for AECOPD Debuts on arXiv
- New arXiv preprint examines Graph-JEPA's category-conditional collapse and proposes diagnostics and repairs
- Software articles
