AviSoori1x/makeMoE

From scratch implementation of a sparse mixture of experts language model inspired by Andrej Karpathy's makemore :)

↗ Open on GitHub

Stars

814

Forks

96

Language

Jupyter Notebook

License

MIT

Last push

2 years ago

Jupyter Notebook llm deep-learning large-language-models neural-networks pytorch mixture-of-experts pytorch-implementation

No related intel yet

This repo has not appeared in any of the sources the radar tracks. The collector runs on a schedule — check back once it covers this repo.