GPU programming is still too hard, and the current LLMs aren't helping much. We think we can build something better, an LLM that can actually write good GPU code.
We're doing this in public - all our training runs, conversations, and infrastructure will be open source.
We mostly talk on discord.gg/gpumode in the popcorn channel. It's pretty casual - drop in when you can.
We're collecting and synthetically generating as many GPU kernels as we can.
We need compute - we're building tools to track progress and share results in real-time through Discord.
We're investigating how to prompt, train, deploy, and sample LLMs for effective GPU code generation.
We're exploring new abstractions to simplify GPU programming such as ThunderKittens.
Our collaborators include
- https://scalingintelligence.stanford.edu/
- https://hazyresearch.stanford.edu/
- https://pytorch.org/
- https://discord.com/invite/gpumode
Our compute sponsors include
And anyone who shares our goals is welcome to join