GitHub - michaelneale/mesh-llm: reference impl with llama.cpp compiled to distributed inference across machines, with real end to end demo
github.com
reference impl with llama.cpp compiled to distributed inference across machines, with real end to end demo - michaelneale/mesh-llm
  • troed@fedia.io
    1·
    11 days ago

    That’s really interesting. Only macOS instructions though? Seems like something that would easily run on Linux as well.

    (I’d love to hook my server’s GPU into local LLM workloads otherwise only offloaded to the CPU from my main workstation when needing too much VRAM)