Exo labs claiming 4.8 tb/s memory bandwidth through m5u Mac Studio clustering

Exo labs making some very exciting and interesting claims. The headline is bandwidth scales linearly on Mac Studio clusters with their solution. There is a thread over at localllm subreddit (https://www.reddit.com/r/LocalLLM/s/qEYLOFaYwc ) where one of their employees speaks about how its latency, not bandwidth that matters in their RDMA clustering solution.

I made a post about m5u 96gb x 2 clustered vs a single m5u 256gb and most folks recommended a single 256, with bandwidth limitations over TB5 being the main reason. I feel like most individuals (myself included) weren’t aware of these claims by Exo when they made those recommendations.

FWIW I think I’m sticking with the 256gb order but I feel like taking the risk on a cluster of 96gb studios is worth considering now given that news. Ultimately I’ll stick with the 256 gb though because in the future it gives me the agility to scale processing power AND ram with a second 256 gb studio if I ever desire it, and maybe I’ll get lucky and buy an off lease unit in 2 years for a lot less, reducing my overall cost per unit ;)