Ello Ello.
Can you recommend a program with which I can generate commit messages locally? I DO NOT want an LLM local or cloud API. What I want is a tiny or small language model. If I could easily swap or customize the model for each project, it would be better, too. So I could have smaller models for specific projects. I can do the training meself.
Thanks in advance!
What is your small lm threshold?
There are Gemma 4 models that run on phones, is that what you’re looking for?
To do this well you will need tool calls because just reading the git diff won’t be enough context to do a good job.
This is probably something you can fine tune, it is getting exponentially cheaper to do this, but without knowing why you don’t want to self host an existing LLM to solve this problem it’s hard to give good advice.
You can look at smollm3 or nanogpt and nanollama as (dated) examples of how to build such a small model yourself, but they’re still LLMs that run on LLM tools. Smollm has guides to fine tune for your task.
You could run them with PyTorch directly or something too, if you’re opposed to the full llama.cpp (vllm is even heavier, with very long startup times).
Thanks but I finally gave up on this :))
The lfm3 models that came out yesterday are 1.7-2b and pretty good, might be worth a try
A commit message is not supposed to be a summary of the code. It is supposed to gives information that you cannot deduct from the diff. Like:
- Is there an attached ticket ?
- Does it produce side effects ?
- Is it a quick and dirty fix just before the release ?
- What are the consequence of the modification ?
- Why you choose this approach instead of another ?
- Why are you not able to add regression test ? etc …
The first guy at Cursor that create the feature to generate commit message from diff is just a fucking moron that don’t understand the role of a commit message.
As a rule, strong feelings about issues do not emerge from deep understanding.
Bullshit. They should also be a summary of the diff. Why would I want to have to read the entire diff to know what the commit is even trying to do?
That’s the role of the commit title, with tags if you follow https://www.conventionalcommits.org/ standard.
99% of commit description will never be read. Only the title is usefull when you scroll to a git log.
But, for the remaining 1%, it means you ended up doing a git blame, git bisect or any funky stuff like this to troubleshot a bug. And when you find the commit that introduced the regression, written by some dude that leaved the company 5 years ago and see a description like “Add the whatever method in whatever class that do the stuff explained by the doc written in the file”… Well, thanks bro. But why did you do this !!!
You might not want to admit it, but you’re asking for a local LLM.
“Small” is just a euphemism for local.
Well, if a meaningful commit message can be created only with a model as big as 1B parameters, I’m better off doing them meself.
“Meaningful” is doing the heavy lifting.
To make a good commit message it must understand your codebase, and programming in general.
You indead have got a point.
Where do you draw the line between Large LM and a non-large LM? You can fit Gemma models into 8GB VRAM or even way less:
How the hell is 8GB a small amount for a side utility? Are you using LLMs to count to 50?
Its small in the sense it’ll run on basically any GPU from the last decade, rather than high end or specialised GPUs. And there is a 1GB gemma4 model, which is meant for CPU/phones.
Well me Intel Arc A380 has got 6GB and I also need to play games…
Gotcha. Sorry for being snarky.
You’re not wrong though, even 1GB is quite lot for writing commit messages, I gave it a quick go, and it takes around 15-30s to analyse a very small diff. Kinda silly for git messages.
