Bursty arrivals speed up LLM inference

(systems.seas.harvard.edu)

2 points | by ak33ra 7 hours ago

1 comments