Architecture Teardown: How Meta Trains LLMs for Code Generation on 100k GPU Clusters
In Q3 2024, Meta trained a 70B parameter code-specialized LLM on 100,000 Nvidia H100 GPUs, achieving 214 TFLOPS per GPU and 92% cluster utilization – …
Architecture
⚑ Report a ProblemLatest Architecture news from Tech News