Just to clear up some potential misunderstanding… Ampere is the Nvidia GPU architecture, and your card is in the same category as RTX30xx. This can matter when it comes to hardware support for llama.cpp and other software.

Having tensor cores affects the speed your card does certain calculations, but AFAIK there are different types of tensor cores in GPUs and if you’re using quantized models (which you most likely will be, unless you’re training your own models) then it’s INT8 performance that matters.

Your amount of VRAM determines how large (which usually correlates to how “intelligent” the model is) model you can fully load on the GPU. If the entire model doesn’t fit in VRAM it can use CPU RAM swapping (partial offloading) which severely harms performance, though MoE models are less affected.

So both VRAM and tensor cores affect generation speed, but in completely different ways.

According to this page ChatGPT would tokenize “seventeen” as [“se”, “venteen”] but AFAIK tokens become multi-dimensional vectors which represents its meaning (all vectors are the same length and the numbers in the vector have nothing to do with the letters that spell it out, just so you don’t imagine something like a C++ string). So an LLM can’t answer how many of each letters there is in a token either, unless explicitly trained to do so. To be able to immediately answer this, I think they would need to include huge amounts of training data to cover all tokens and letter combinations.

But I would have expected that since the strawberry incident, all models would have been trained to do what Qwen tried to do here, and break the word down into letters. Because “s e v e n t e e n” tokenizes to [“s”, “e”, “v”, “e”, “n”, “t”, “e”, “e”, “n”] so then they only have to train the model to be able to convert any token into its individual letter, and to count occurrences of something in an array.

However, if you disallow thinking and force the LLM to answer immediately, then you’re back to the original problem and will likely get an incorrect answer.

From what I have found, the main difference between an eBPF program (which is what EACv2 contains) and an actual kernel module, seems to be that other parts of the kernel will be protected from EACv2. However, an eBPF program can spy on all your user space programs, read memory belonging to other processes, etc. They’re also saying that the anti-cheat is going to include a systemd service, which I assume is going to run as root.

EAC v2 is technically not in the kernel, it uses existing Linux security features to get the same level of control over your computer AFAIK. Which is probably what Microsoft is aiming for too

Well, it’s just a guess. But also Gemma4 increased to 31B, so I think there’s a possibility that Qwen does the same.

27B dense was a good size for 3.8, but I’m guessing they’re going to add a few B parameters in order to make it noticeably better.