My hobby project gets much less of my time than I would prefer. I've got the strongest model I have access to, on a cheap subscription, running --yolo in a VM with a fairly locked down firewall. I found that a tree (i.e. the filesystem) isn't really enough, for representing my thoughts, so I've built a graph datastore of sorts with a query layer that basically consists of a view (fitting the graph onto a tree) and a filter (for the root of that tree). Technically I've got a review process for the graph, but in practice I just approve it. I'm planning to build out an automated review process, kinda like unit tests, where I give a given prompt ownership over a specific cluster of nodes (and the code paths owned by them), and that prompt has to approve changes before I see them, but I haven't done that yet.
To answer your question, the actual review I do, is when the agent asks me questions. I tell it to resume a task, it asks questions, I answer them. If those questions are off-the-rails, I stop, pop the conceptual stack, and come at the whole thing from a different angle.
I'm basically trusting the model to do a good job, and fixing forward, always annealing the project toward what I want. And the rejection criteria is when the questions that it's asking are _questions I don't care about_.
But if you never review it, how do you know it’s correct, secure, maintainable, performant, etc? You just trust it’s correct because you told the agent to make it correct?
My hobby project gets much less of my time than I would prefer. I've got the strongest model I have access to, on a cheap subscription, running --yolo in a VM with a fairly locked down firewall. I found that a tree (i.e. the filesystem) isn't really enough, for representing my thoughts, so I've built a graph datastore of sorts with a query layer that basically consists of a view (fitting the graph onto a tree) and a filter (for the root of that tree). Technically I've got a review process for the graph, but in practice I just approve it. I'm planning to build out an automated review process, kinda like unit tests, where I give a given prompt ownership over a specific cluster of nodes (and the code paths owned by them), and that prompt has to approve changes before I see them, but I haven't done that yet.
To answer your question, the actual review I do, is when the agent asks me questions. I tell it to resume a task, it asks questions, I answer them. If those questions are off-the-rails, I stop, pop the conceptual stack, and come at the whole thing from a different angle.
I'm basically trusting the model to do a good job, and fixing forward, always annealing the project toward what I want. And the rejection criteria is when the questions that it's asking are _questions I don't care about_.
Not supposed to edit or touch any code, or review. Other agent reviews and approves. You only write prompt.
But if you never review it, how do you know it’s correct, secure, maintainable, performant, etc? You just trust it’s correct because you told the agent to make it correct?
I guess so, more AI is always a solution. I still like to review what the agents produce from time to time.