No. of Recommendations: 11
My biggest concern is that their profit margins are sufficiently obscene that their very very rich clients have a huge motivation to roll their own. That would take a lot of time and money, but doesn't seem impossible.Already happening; and its taking lots of capital but Google is making in roads with its TPU chips.
https://news.ycombinator.com/item?id=46069048Quote:
Google's real moat isn't the TPU silicon itself—it's not about cooling, individual performance, or hyper-specialization—but rather the massive parallel scale enabled by their OCS interconnects.
To quote The Next Platform: "An Ironwood cluster linked with Google’s absolutely unique optical circuit switch interconnect can bring to bear 9,216 Ironwood TPUs with a combined 1.77 PB of HBM memory... This makes a rackscale Nvidia system based on 144 “Blackwell” GPU chiplets with an aggregate of 20.7 TB of HBM memory look like a joke."
Nvidia may have the superior architecture at the single-chip level, but for large-scale distributed training (and inference) they currently have nothing that rivals Google's optical switching scalability.
tecmo
...
