I haven't noticed any difference. In fact, I just set it on a laundry list of 12 items to sort out overnight. It took 10% of my weekly quota to do it, and it probably would have only taken half the tokens if I'd been monitoring it. But it did work while I slept, so I'm not too torn up about it.
There were a few small issues with its work (it's not the greatest at getting layouts right, and one of the features it implemented feels clunky and will need tuning), but the architectural refactors it did worked perfectly and I'm happy with the performance improvements.
I haven't yet checked out the research tasks it did, but usually it does a decent job. I could have just asked it to research and implement, but I find it best to manually verify its conclusions first.
Well the biggest disconnect is that I rarely run the orchestrator model at medium -- I just checked the ArtificialAnalysis index, and Opus-5.5-xhigh is about as much better than Opus-5.5-medium than Opus-5.5-medium was than Opus-5-medium (~5 PP, or ~10%). If I'm letting a model plan changes and then orchestrate their implementation without talking through the details first, it's almost always worth it to me to pay for that extra intelligence.
The second biggest difference, though, is that I haven't used Opus 5.5 in a few days cause I ran out of credits on Thursday! So... you might be doing something right, there ;)
One thing you could check is the files that tend to get read into context, especially their built-in "memory" feature, which can get insanely cluttered if not actively maintained. This is why I prefer keeping everything I can to three other abstractions, none of them harness-specific: reports (roughly per-(long-)turn), policies (human readable, very concise, identified with unique slugs, system-wide), and tasks (Plane/Trello/Linear/etc., mentioned in git commits).
For something like "high level performance choices", a policy to always think about performance explicitly before reporting back could honestly do wonders. As you say yourself, they're typically smart enough to know what to do when reminded of your priorities or asked explicitly. The trick is to not prioritize everything, everywhere, all at once!
...so weird talking about how to best use these agents, BTW. Truly shuffling the deck chairs on our Titanic of a career/civilization.
I haven't noticed any difference. In fact, I just set it on a laundry list of 12 items to sort out overnight. It took 10% of my weekly quota to do it, and it probably would have only taken half the tokens if I'd been monitoring it. But it did work while I slept, so I'm not too torn up about it.
There were a few small issues with its work (it's not the greatest at getting layouts right, and one of the features it implemented feels clunky and will need tuning), but the architectural refactors it did worked perfectly and I'm happy with the performance improvements.
I haven't yet checked out the research tasks it did, but usually it does a decent job. I could have just asked it to research and implement, but I find it best to manually verify its conclusions first.
Well the biggest disconnect is that I rarely run the orchestrator model at medium -- I just checked the ArtificialAnalysis index, and Opus-5.5-xhigh is about as much better than Opus-5.5-medium than Opus-5.5-medium was than Opus-5-medium (~5 PP, or ~10%). If I'm letting a model plan changes and then orchestrate their implementation without talking through the details first, it's almost always worth it to me to pay for that extra intelligence.
The second biggest difference, though, is that I haven't used Opus 5.5 in a few days cause I ran out of credits on Thursday! So... you might be doing something right, there ;)
One thing you could check is the files that tend to get read into context, especially their built-in "memory" feature, which can get insanely cluttered if not actively maintained. This is why I prefer keeping everything I can to three other abstractions, none of them harness-specific: reports (roughly per-(long-)turn), policies (human readable, very concise, identified with unique slugs, system-wide), and tasks (Plane/Trello/Linear/etc., mentioned in git commits).
For something like "high level performance choices", a policy to always think about performance explicitly before reporting back could honestly do wonders. As you say yourself, they're typically smart enough to know what to do when reminded of your priorities or asked explicitly. The trick is to not prioritize everything, everywhere, all at once!
...so weird talking about how to best use these agents, BTW. Truly shuffling the deck chairs on our Titanic of a career/civilization.
[flagged]
[flagged]