• 0 Posts
  • 77 Comments
Joined 2 years ago
cake
Cake day: June 4th, 2023

help-circle


  • Not sure if it counts as “budget friendly” but the best and cheapest method right now to run decently sized models is a Strix Halo machine like the Bosgame M5 or the Framework Desktop.

    Not only does it have 128GB of VRAM/RAM, it sips power at 10W idle and 120W full load.

    It can run models like gpt-oss-120b or glm-4.5-air (Q4/Q6) at full context length and even larger models like glm-4.6, qwen3-235b, or minimax-m2 at Q3 quantization.

    Running these models is otherwise not currently possible without putting 128GB of RAM in a server mainboard or paying the Nvidia tax to get a RTX 6000 Pro.











  • I use Jellyfin but I download all my songs from Tidal, Qobuz or Deezer and tag them automatically right then and there in a clean format so Jellyfin does not have to guess at all.

    I also have some automatic checks in place to convert incorrect metadata to a proper format. Like moving artists from the title (feat. Somebody else) to the artists tag Somebody; Somebody else and a bunch more.

    Together with Finamp on desktop and mobile everything is pretty much working as expected.


  • I’m running this on a 7900 XTX with 32GB RAM. No issues so far. According to their instructions, Nvidia is a little bit more involved but it should perform the same on consumer or pro GPUs.

    I assume decause it’s using Docker, the more RAM the better.

    Docker has pretty much no overhead, so you only need enough RAM to run the games/sessions you want to run in addition to your regular desktop.


  • They don’t do the same thing: Sunshine is intended to stream a single physical desktop.

    Games on Whales runs headlessly and creates virtual desktops for each session in a Docker environment.

    For example, you can create an instance that runs at 800p so you can stream to your Steam Deck at its native resolution. You can even still use your desktop normally since the streams run in the background.

    Both of them support connection via Moonlight.