The first one I've seen (although I'm sure there are and will be more as more time progresses...)
It seems to require a bit more system memory (70GiB free RAM as opposed to 32GB) and two newer nVidia RTX 50 series graphics cards with 16GB (as opposed to a larger choice of multi-vendor older single graphics cards with a minimum of 12GB), but on the flip side, it does seem to make up for that with faster token generation speed...
Hi thx for the reply. Capy works on single card also, but yes the optimisations are done for double card and for nvfp4 nvidia format, that's native on blackwell, so this can also be used in newer datacenter gpus. On the right setups it's faster than strata
A Strata (https://github.com/Niko1221/Strata) challenger!
The first one I've seen (although I'm sure there are and will be more as more time progresses...)
It seems to require a bit more system memory (70GiB free RAM as opposed to 32GB) and two newer nVidia RTX 50 series graphics cards with 16GB (as opposed to a larger choice of multi-vendor older single graphics cards with a minimum of 12GB), but on the flip side, it does seem to make up for that with faster token generation speed...
Anyway, it seems to be worth watching...
Strata may have a worthy competitor here!
Hi thx for the reply. Capy works on single card also, but yes the optimisations are done for double card and for nvfp4 nvidia format, that's native on blackwell, so this can also be used in newer datacenter gpus. On the right setups it's faster than strata