• 1 Post
  • 38 Comments
Joined 1 year ago
cake
Cake day: August 26th, 2025

help-circle








  • Really depends on how much headroom you need and your system ram and the PCIe / vram speed because I got a friend with a pcie5 16gb gpu and 64gb of system ram that’s getting slightly better performance than me using the same setup, despite half the vram.

    If you want speed and not a ton of intelligence, the 3.6 30B A3B runs fast. If you need smarts, then either a qwen 3.8 27b (probably a thinking cap finetune) or try flash next (swift finetune) in unbiased strata if you don’t mind taking a risk and burning some bandwidth on a 75-85GB model that might not run great. In either case, be sure to use MTP and the model card inference params





  • I built my custom solution with a shared systemd qemu VM service to run harnesses isolated from my host that mount specific directories and run via wrappers that start and automatically SSH into them, as well as native variants that run on the host machine. (Plus args for quick selection and feature toggling e.g. “cx sol high memory” as well as per-harness hooks for my agnostic skill-like text injection system, memory tools, optimized system prompts, etc)

    All of this is done in my nix flake (but can’t share here)

    For Deepseek itself, I’m just using Moraxyc/deepseek-harness.nix for the package, and using its webui, with wrappers to integrate it with my vm/native dichotomy system.

    I haven’t done any deep tooling integration or prompt rewriting with dsh like I have with codex and claude code. It’s still pretty early days for me using it and I prefer terminal apps in a multiplexer like herdr. But it doesn’t seem to need it so much either: sonnet eating 35k tokens for a “hello” out of the box was egregious to me, but dsh seems to be more light so I haven’t felt the need, and I don’t do as much with it and haven’t used it for very long yet so I haven’t felt the need to customize it as much as I have the other harnesses… Which is a bit ironic given that its customization and flexibility are pretty much its main selling points.



  • Well, I think a caveat to that is publishing source (no prebuilt release) as unvetted slop, so everybody knows. Though in this case, it was strictly for personal use.

    But yes. Anything I release proper, for general consumption, is owned by me. And professionally, I never own even a line of code (or comment) that I don’t understand or wouldn’t have written myself. I may ask for a bunch of audits and slop tests for insights, but I’ll keep those offline as “extra” unless it’s something on par with I would have written. I’ll never hallucinate important code, and anything i get paid for or contribute to OSS is important. Personal game mods and small utilities, not so much.