https://www.msuiche.com/tags/h100/Recent content in H100 on Matt SuicheHugoen-usTue, 30 Sep 2025 00:00:00 +0000https://www.msuiche.com/posts/optimizing-alphafolds-triangle-multiplicative-update-a-first-look-at-gpu-performance-engineering/Tue, 30 Sep 2025 00:00:00 +0000https://www.msuiche.com/posts/optimizing-alphafolds-triangle-multiplicative-update-a-first-look-at-gpu-performance-engineering/<h2 id="background">Background<a href="#background" class="anchor" aria-label="Link to Background">#</a></h2> <p>I recently encountered the <a href="https://www.gpumode.com/v2/leaderboard/496?tab=submission" target="_blank" rel="noopener">GPU MODE TriMul challenge</a> while exploring GPU optimization. Coming from a systems engineering background without prior PyTorch or Triton experience, this challenge provided an opportunity to learn GPU performance engineering through a practical problem.</p> <p>The Triangle Multiplicative Update (TriMul) is a core operation in AlphaFold2 and AlphaFold3—the protein structure prediction systems that earned the 2024 Nobel Prize in Chemistry. The operation&rsquo;s O(n³) complexity creates severe performance bottlenecks in production, forcing AlphaFold3 to use batch size 1 during training despite having under 1B parameters. This makes the optimization problem both practically relevant and technically challenging.</p>