14 points | by vatsalaggarwal 7 hours ago
2 comments
enjoyed testing out Leo & Mia. Exciting mission - wondering what the benchmarks are like?
glad to hear it!
> wondering what the benchmarks are like?
we've avoided OS benchmarks as they don't quite represent the scenarios and problems our customers face.
we instead developed an internal eval set that more closely represents our customers workflows / domain.
enjoyed testing out Leo & Mia. Exciting mission - wondering what the benchmarks are like?
glad to hear it!
> wondering what the benchmarks are like?
we've avoided OS benchmarks as they don't quite represent the scenarios and problems our customers face.
we instead developed an internal eval set that more closely represents our customers workflows / domain.