Grok4 was trained on 100k or 200k GPUs (as far as I understand) Grok5 might need... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		MichaelRazum 10 months ago \| parent \| context \| favorite \| on: Zuckerberg says Meta will build a data center the ... Grok4 was trained on 100k or 200k GPUs (as far as I understand) Grok5 might need 1MM or 2MM. So the question is what about metas / zucks plans? How many GPUs will Manhattan get? Looks like, that to get the next unlock you need crazy amounts of compute.

jiggawatts 10 months ago [–]

Meta had the equivalent of about 600K H100 cards a year ago, but they were geographically distributed and used mostly for inference.

These giant data centres will allow these companies to put about a million in one location and possibly into a single giant training cluster.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact