7 points | by gurveer51 9 hours ago
1 comments
Curious how semantic cache works,are cache hit based on embeddings, exact prompt normalization, how do you avoid serving stale incorrect cache..
Curious how semantic cache works,are cache hit based on embeddings, exact prompt normalization, how do you avoid serving stale incorrect cache..