pjlab-sys4nlp/llama-moe

⛷️ LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training (EMNLP 2024)

↗ Open on GitHub

Stars

1,002

Forks

61

Language

Python

License

Apache-2.0

Last push

2 years ago

Python llm llama mixture-of-experts moe continual-pre-training expert-partition

No related intel yet

This repo has not appeared in any of the sources the radar tracks. The collector runs on a schedule — check back once it covers this repo.