This person on huggingface makes excellent space saving models using APEX quantizations. They produced this one to fit on gpus with 16gb of vram or less.
- 0 Posts
- 4 Comments
Joined 3 years ago
Cake day: June 16th, 2023
You are not logged in. If you use a Fediverse account that is able to follow users, you can follow this user.
How did you configure DeepSeek harness? I’m guessing a flake would be useful.
I read your post and ended up finding this repo: https://github.com/numtide/llm-agents.nix
I’m considering deploying it for testing in a nix container. 🤔
Oh that sounds rad lol. I was considering the possibility of building a setup within nixos using vfio to declare and configure VMs. I like your integration with systemd, that’s clever