Posts Tagged "quantization"
GLM-5.2 NVFP4: fast, cheap, and not worth serving
We put an NVFP4 build of GLM-5.2 in front of real users for four days, at 200+ tokens per second. Users called it intoxicating. We retired it anyway: if the tokens are not useful, they are not worth serving, no matter how efficiently we can produce them.
Read Post