NUMA: Cores, memory, and the distance between them

2 comments

• lukax 2 hours ago
  NUMA can cause really crappy performance. We deployed a Go based LLM gateway in Kubernetes deployed on a server with hundreds of CPU cores. We didn't explicitly set GOMAXPROCS so Go runtime scheduled goroutines over different CPUs and it constantly used 200% CPU and GC was causing latency spikes. Then we set GOMAXPROCS 8 and all performance issues went away. Until recently Kubernetes didn't work well with NUMA.
  [-]
  • CarRamrod 21 minutes ago
    There is one instance where the NUMA performance never disappoints: https://www.youtube.com/watch?v=Cqd1Gvq-RBY
  • re-thc 1 hour ago
    Is this on AMD? I wonder if it's all to do with NUMA or their CCD architecture etc (well these days Intel and everyone also does it to some extent).
    [-]
    • toast0 1 hour ago
      Hundreds of cores is likely two sockets and so you've got NUMA there.

      Scaling to large core counts has a lot of gotchas.
• StreamCtx 1 hour ago
  [dead]