Interesting! Sort of a tangential question to the product itself but why did you choose to build primarily in Rust? I saw "built with Rust + simd-json + ratatui for ultra-fast performance" but am not actually very familiar with how these technology choices specifically enable ultra-fast performance. Would love to learn more!
Good question! Honestly, the README is a bit misleading - ratatui is just for the terminal UI, not performance. Let me clarify
The actual speed comes from
1.simd-json: This is the big one. It uses CPU SIMD instructions to parse multiple JSON bytes in parallel at the hardware level. We're talking ~3 GiB/s vs ~300 MB/s with standard parsers.
2.rayon: Dead simple parallel processing. Instead of parsing 2,000 files one by one, it spreads them across all CPU cores.
3.Rust itself: No GC means no random pauses when you're crunching through gigabytes of data. The original Node.js version would just... freeze sometimes.
The 40s → 0.04s improvement basically "what if we actually used the hardware properly?" SIMD for parsing, all cores for parallelism, no GC getting in the way.
(I should probably fix that README line - thanks for pointing it out!)
Interesting! Sort of a tangential question to the product itself but why did you choose to build primarily in Rust? I saw "built with Rust + simd-json + ratatui for ultra-fast performance" but am not actually very familiar with how these technology choices specifically enable ultra-fast performance. Would love to learn more!
Good question! Honestly, the README is a bit misleading - ratatui is just for the terminal UI, not performance. Let me clarify
The actual speed comes from
1.simd-json: This is the big one. It uses CPU SIMD instructions to parse multiple JSON bytes in parallel at the hardware level. We're talking ~3 GiB/s vs ~300 MB/s with standard parsers.
2.rayon: Dead simple parallel processing. Instead of parsing 2,000 files one by one, it spreads them across all CPU cores.
3.Rust itself: No GC means no random pauses when you're crunching through gigabytes of data. The original Node.js version would just... freeze sometimes.
The 40s → 0.04s improvement basically "what if we actually used the hardware properly?" SIMD for parsing, all cores for parallelism, no GC getting in the way. (I should probably fix that README line - thanks for pointing it out!)