The top 3 repos sorted by stars on Huawei's github page are all for quantization. I am not an LLM scientist so I can't tell what the difference is between them, or why they need three different pipelines for model quantization.
SINQ was created last year. It was for weights only and it was calibration-free quantization. Then KVarN is for KV-cache quantization to get longer context. MACaQ is multimodal, activation and weights calibrated quantization. 3 different projects
Why does some HN users seem to prefer GitHub page whenever available at expense of official site? I mean, Huawei is still sanctioned, anything they host third party outside China can disappear at any hour.
I prefer GitHub, because I can see the code and immediately know where to click. Product pages are always full of various marketing shots, videos, etc that just takes more time to parse through.
People are reproducing it and creating GGUFs https://www.reddit.com/r/LocalLLaMA/s/36atJ2nNjh
The results shown there are very good, much better than for the previously widely used quantizers.
Nice for Huawei to share it.
Sorry, I've deleted my Reddit account since they locked out users. Could you please post links to the GGUFs here :)
The top 3 repos sorted by stars on Huawei's github page are all for quantization. I am not an LLM scientist so I can't tell what the difference is between them, or why they need three different pipelines for model quantization.
SINQ was created last year. It was for weights only and it was calibration-free quantization. Then KVarN is for KV-cache quantization to get longer context. MACaQ is multimodal, activation and weights calibrated quantization. 3 different projects
https://huawei-csl.com/blog/macaq/
Why does some HN users seem to prefer GitHub page whenever available at expense of official site? I mean, Huawei is still sanctioned, anything they host third party outside China can disappear at any hour.
I prefer GitHub, because I can see the code and immediately know where to click. Product pages are always full of various marketing shots, videos, etc that just takes more time to parse through.
Looks like many of their repos there have been around for a while. Also not sure why we would sanction free stuff from them
Did they release any model? Seems good for Strata
Qwen 3.8 27B has a macaq quant available
What a project name.
Probably intended to reference macaque monkeys.
xD
First thing my mind went to.
"Huawei releases MACaQ"
"Just like Jensen released his..."
I do not think that macaques would mind.
Just enough plausible deniability :P