b4rtaz/distributed-llama

Distributed LLM inference. Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference.

↗ Open on GitHub

Stars

3,050

Forks

248

Language

C++

License

MIT

Last push

2 months ago

C++llm llm-inference neural-network llama3 llms llama2 open-llm distributed-computing distributed-llm

No related intel yet

This repo has not appeared in any of the sources the radar tracks. The collector runs on a schedule — check back once it covers this repo.