This fork of antirez/ds4 is a focused, measured configuration for running DeepSeek V4 Flash on an NVIDIA GeForce RTX 5080 with 16 GB of VRAM, using CUDA and a fast NVMe SSD. Development and performance measurements were made on an RTX 5080 Laptop GPU (Blackwell, sm_120) under Linux/WSL2.
This fork of antirez/ds4 is a focused, measured configuration for running DeepSeek V4 Flash on an NVIDIA GeForce RTX 5080 with 16 GB of VRAM, using CUDA and a fast NVMe SSD. Development and performance measurements were made on an RTX 5080 Laptop GPU (Blackwell, sm_120) under Linux/WSL2.
3.7tok/s decode ain't bad at all
for that setup i mean
Yes it's the best performance i can reach with a lot of caching between network layers of experts. But is incredible to make it run at all !!!