We ended up doing the same, own harness with a world view (schema.org) and map reduce rather than a RAG. Happy with the results and context hydration speed
Author here. This post covers how we rebuilt our agent after a tool-per-resource approach didn't scale — we replaced most of our tools with a virtual filesystem the agent explores via bash.
A few things that might be interesting to discuss:
- We didn't want to boot a container per session, so we run a bash interpreter + virtual FS in-memory as a process in our Elixir cluster. This is a port of Vercel's just-bash (TypeScript) to Elixir. The original's test suite made it a well-defined target for an agent-assisted port — we reused the fixtures verbatim: https://github.com/elixir-ai-tools/just_bash
- The "why not a real sandbox" tradeoff is the one I'm least certain about long-term. In-memory gets us instant starts and no sync problems, but if we add a real scripting language (Python) for the agent, we'll probably have to swap in a real sandbox. We've kept the interfaces decoupled so that swap stays cheap.
- For data that doesn't map well to static files (logs, message history), we registered a `knock` CLI inside the bash environment instead of adding more tools. Because it has --help for every resource, the agent learns it with almost no steering.
Happy to answer questions about the architecture, the Oban-backed agent loop, or how we're evaluating trajectories (nightly evals, pass^k on common tasks).
This is probably naive of me, but avoidance of a real sandbox sounds like a great idea so far:
1: Apply the Principle of Least Power/Privilege, especially where agents are involved. "With great power comes astonishing fuckupitude."
2: More freedom to diverge from the existing POSIX and bash primitives to something that better suits the data involved
Having logical control of the filesystem lets you do fun tricks like using file paths as a simplistic search API (e.g. /docs/tag1/tag2/ lists all the docs tagged with both tag1 and tag2).
But it might also be worth looking at shells which give you more power over structured data, like nushell. It's written in Rust, but it might be relatively simple to port just the bits you need to Elixir (which I would never have said a couple of years ago). Keeping everything in process sounds like a major win to me, because of both performance and control.
We ended up doing the same, own harness with a world view (schema.org) and map reduce rather than a RAG. Happy with the results and context hydration speed
I thought it would be bash+fuse but it's actually running in the matrix: simulated bash in Elixir ported from Vercel's just-bash:
> A virtual bash environment with an in-memory filesystem, written in TypeScript and designed for AI agents.
>Broad support for standard unix commands and bash syntax with optional curl, Python, JS/TS, and sqlite support.
https://github.com/vercel-labs/just-bash/tree/main/packages/...
See also (credited by TFA):
https://vercel.com/blog/how-to-build-agents-with-filesystems...
What harness are you guys running?
FTA: "To bring this vision to life, we needed to create a rich agent harness"
sounds to me like they built their own
Author here. This post covers how we rebuilt our agent after a tool-per-resource approach didn't scale — we replaced most of our tools with a virtual filesystem the agent explores via bash.
A few things that might be interesting to discuss:
- We didn't want to boot a container per session, so we run a bash interpreter + virtual FS in-memory as a process in our Elixir cluster. This is a port of Vercel's just-bash (TypeScript) to Elixir. The original's test suite made it a well-defined target for an agent-assisted port — we reused the fixtures verbatim: https://github.com/elixir-ai-tools/just_bash
- The "why not a real sandbox" tradeoff is the one I'm least certain about long-term. In-memory gets us instant starts and no sync problems, but if we add a real scripting language (Python) for the agent, we'll probably have to swap in a real sandbox. We've kept the interfaces decoupled so that swap stays cheap.
- For data that doesn't map well to static files (logs, message history), we registered a `knock` CLI inside the bash environment instead of adding more tools. Because it has --help for every resource, the agent learns it with almost no steering.
Happy to answer questions about the architecture, the Oban-backed agent loop, or how we're evaluating trajectories (nightly evals, pass^k on common tasks).
This is probably naive of me, but avoidance of a real sandbox sounds like a great idea so far:
1: Apply the Principle of Least Power/Privilege, especially where agents are involved. "With great power comes astonishing fuckupitude."
2: More freedom to diverge from the existing POSIX and bash primitives to something that better suits the data involved
Having logical control of the filesystem lets you do fun tricks like using file paths as a simplistic search API (e.g. /docs/tag1/tag2/ lists all the docs tagged with both tag1 and tag2).
But it might also be worth looking at shells which give you more power over structured data, like nushell. It's written in Rust, but it might be relatively simple to port just the bits you need to Elixir (which I would never have said a couple of years ago). Keeping everything in process sounds like a major win to me, because of both performance and control.