so you're right its not exactly the same (and it might not even be a very similar implementation ala how docker is), but the bullet points are the same.
ephemeral, instant boot strap, append only (I'm pretty sure I was the first person to use ephemeral computing this way, at the time I couldn't find anything that touched it. People used containers, ala ZFS/Zones, but they were generally meant for long term computing and not for bringing up an environment for a very small period of time and then tearing it down.)
though we also approached recording and forking your computation at will - https://dl.acm.org/doi/10.1145/1323293.1294289 (I'm proud of coining the name), though it strongly relates to Microsoft's recall so it would probably be viewed less positively today (in fact, this might be closer in implementation to what they are doing than the above).
with that said, it was a bit more tongue in cheek than anything.
as an aside, I sort of wonder in the age of AI and the ability to reproduce anything easily, we might have to expand patents (related to comment below that got downvoted), but dont have strong thoughts on this yet.
Folks might also be interested in try [1].
Lets you run anything (including your agent) in a mirror of your current system and then inspect/apply/discard/++ any of its effects.
The evergreen observation; The most self-promote-y among us are mostly using AI to reinvent solutions and disguise them as new designs. These address solved problems, which are themselves disguised as new agentic problems.
Celebrating and promoting this stuff does not help us get better at discovering and building actually new things; change my mind.
It is more content aware and tuned towards versioned data vs block aware among other things. There’s a benchmark as to how it impacts things in the site
I think it’s actually a fairly useful thing but I worry about something so obviously vibe coded hitting 1.0.0 and without a track record but the ideas seem pretty useful. I’ve certainly found the work tree semantics most agents and harnesses follow at scale to be obnoxious at best and it seems to tackle issues head on.
Nice benchmarks, impressive numbers. 20x less space than btrfs for the deep seek harness.
They even have some explanations. Files writes in btrfs snapshots recreatethe contents as opposed to storing what's changed. Btrfs has no dedupe, if multiple snapshots happen to have the same data. And small files store metadata twice in btrfs for reasons.
I do think their use case is a bit pathological, checking out the same harness at every commit. Most of these btrfs gotchas don't feel like they would be super impactful to me. But really cool results none the less.
I have this pattern but because I run pipelined agents working on a speculative merge topologically sorted task list merging into a merge train. This leads to lots and lots of work trees that largely overlap except for slight variants. Their articulated use case is mine, and I can have up to a hundred work trees at a time. The storage and related back pressures can be complicated.
For the source code part of that, the difference between using layerfs and btrfs snapshots feels near trivial. There'd be so little difference at the source layer. Unless your changes are massive massive massive repo wide changes, most of the content is going to remain the same, and the btrfs snapshot is going to mean each workspace is a proportionally a tiny delta, have as many as you want.
I guess the win is that often build artifacts are the same in many cases, so a pristine checkout still de-dupes. There's very good build caching systems out there that help with not needing to keep re-compiling the exact same stuff you've compiled, so I tend to discredit this usage some. But each of those workspaces would end up with it's own copy of the build artifacts, so there is a real savings to layerfs in either case, with it's dedupe.
Good for you on your "speculative merge topologically sorted task list merging into a merge train". Fun hard problems to have! Great. I don't want to be too rude, but also, personally, I think you could have simmered that down a bit, at the cost of not presenting quite entirely how super advanced elite super cool you are.
Perhaps but it wouldn’t have been explicit about why I have that problem. I don’t think it’s a normal problem to have, whether it’s elite or not, it’s IMO not common. The speculative pipeline and the merge train is the key part because they overlap with changes in the same files. This is why block based doesn’t help me as much. I’d prefer to not censor myself with respect to my problem statements for some internet humility virtue signaling.
"I have lots of builds on the same tree" is not really losing much relevant signal. I don't think I'm trying to make you earn "virtue signaling" humility points, I'm trying to increase the chance people understand and ascertain what you're actually saying. Without having to spend time digging through all the inflated noise you're giving off. Communication and brevity are worthy objectives.
If your commitment is just to bragging keep doing what you're doing. I'm just trying to help you be effective. Apologies, I guess? You seem not only not interested, but you're showing some gross colors in the process (virtue is good, actually).
As much as I appreciate your intention might be to be helpful it was in fact quite rude and arrogant. I think telling someone in a technical forum to not be so technical and lecturing them on clarity of communication is pretty offensive - you don’t know me, know what I do or who I am, what my background is, and clearly my message wasn’t intended for your consumption since you found it not for your consumption. Some advice since we are dishing it out, this sort of stuff is as annoying as correcting grammar on Usenet posts or all the other forms of things that disingenuously begin with “not to be rude” - if you are aware it is likely to be read as rude, maybe just not type it next time? Just trying to help you be effective and not show such gross colors next time.
PS, it’s about builds on the same tree but about edits on the same set of files in the tree that would cause block COWs in brtfs. If you read the original links docs you will see that’s the use case they’re targeting, and it’s a lot different than build artifacts. Their proposal isn’t just assisting in deduping but in merging rapidly files that have interleaved changes.
I started on a bad foot when you messaged me, and here's why it rubbed me wrong: it seemed like btrfs snapshots would give you 98% of the wins just as easily. With far better reliability than using a nosync sqlite as a backing store. And I'd talked thought some of the differences. But it didn't seem like you were connecting with or going further into any of the material or tradeoffs or how the two would meet your workload. You didn't connect your work to the subject at all other than to show up and plant a flag. I found it rude and arrogant that you'd just show up to share your thing and be so technically disconnected from the topic.
That all made me pretty sore. Complaining about virtue signalling & this post here, well, we're done here. I don't feel like you've done anything to connect me with the problems. Btrfs seems like it would work just fine for your use cases. You haven't talked to the problem or drawn any connection. Not really interested anymore. Good luck.
The difference is this one (layerfs) was vibe coded™ and the other (btrfs) is battle tested in mission critical systems for years in the Linux kernel and used by many companies in the SNP 500.
Do not be fooled by the "vibes" of the website design which many here are. If it is too good to be true, it most likely is.
That's what I was concerned about. An unstable filesystem is worse than no filesystem at all; I'd really like to switch to bcachefs but it simply needs to have a few more years under its belt before I'll be comfortable with it (particularly for some of its best use cases, where it becomes responsible for backups).
It's fine if you stay away from it's raid5/6. And realize it isn't going to let you know when you have a mirror fail until you try to boot it and it won't come up.
I disagree. Its full of AI slop. Too much animation (cute but distracting) I literally tried for 20 minutes and couldn't focus on reading. I just couldnt comprehend anything.
~245k lines of Rust the LLMs coded. So my question is other than DSH does the author(s) have any advice on setup that made creating this possible? or other have tips for best setup that can produce a project like this.
Even those bashing for being AI generated have to admit it’s technically impressive and reflective of where software engineering is headed.
I have to try it but I am very well interested in the decentralized agents mentioned there, I am tired of scattered agents and sessions across multiple OSes and harnesses I have, I just need one say an LXC that I can self host and automatically sync (or rather async like push/pull) all my harnesses, regardless wherever the previous session was.
You can achieve this by externalizing the memory outside of every harness and then connecting all harnesses to it via MCP.
Then you will have a single store where you can both read and write, and nothing will need to be synced.
I am the founder of Substrate, a project that aims to provide this capability: having just one memory for all your agents, sessions and tools.
What two harnesses do you switch between most often?
Not memory, rather the sessions and skills. So you start your session in opencode on linux with say sol model, then the next hour you pull your Mbp start claude code on macos and cont the session on opus, with the same skills too. Memory isn’t that useful I found it actually bad as it impacts the output in a different direction than what was asked. I was thinking to build it but didn’t have the time, if something proven exists and open source (to trust it with sessions) then that would be great.
If the skill wa already loaded into context tranplanting it into a new harness will move that skills context as well (not the scripts though).
What kind of memory has degraded your performance?
Memory is quite tricky to tune and simple a direct implementations of an LLM Wiki concept can indeed have a lot of problems, as context accumulates.
> start your session in ... then ... cont the session on
This is not a technical restriction, but a commercial one. Any vendor agnostic harness can implement session sharing as a feature, but because you would be made to pay higher token pricing, there will be little interest among the consumers.
You can do session sharing in proprietary harnesses as well - we do that at Substrate.
Sessions are just text - and can be manipulated however you want.
lmk if you want a quick onboarding to trysubstrate.co
Been there (16 years ago), done that (15 year ago), patented it (13 years ago) :) almost the same name too.
https://www.usenix.org/conference/usenix-atc-10/apiary-easy-...
https://www.usenix.org/legacy/event/lisa11/tech/full_papers/...
https://patents.google.com/patent/US8589947B2/
Would you mind ELI5’ing the similarities? I’m seeing only high level concepts that sound the same.
so you're right its not exactly the same (and it might not even be a very similar implementation ala how docker is), but the bullet points are the same.
ephemeral, instant boot strap, append only (I'm pretty sure I was the first person to use ephemeral computing this way, at the time I couldn't find anything that touched it. People used containers, ala ZFS/Zones, but they were generally meant for long term computing and not for bringing up an environment for a very small period of time and then tearing it down.)
This paper with bellovin applied it to system administration - https://www.usenix.org/legacy/event/lisa09/tech/full_papers/...
though we also approached recording and forking your computation at will - https://dl.acm.org/doi/10.1145/1323293.1294289 (I'm proud of coining the name), though it strongly relates to Microsoft's recall so it would probably be viewed less positively today (in fact, this might be closer in implementation to what they are doing than the above).
with that said, it was a bit more tongue in cheek than anything.
as an aside, I sort of wonder in the age of AI and the ability to reproduce anything easily, we might have to expand patents (related to comment below that got downvoted), but dont have strong thoughts on this yet.
All good points. Related to your observation about patents, well, I guess we'll have to make patent descriptions even more inscrutable. ;-)
I wonder why their AI coding agent didn't flag the patent.
[dead]
Here’s an alternative that runs on your browser: https://nicobrenner.github.io/lossy-git/demo.html
PS: it’s just a fun play on memory/storage and AI. Hope you enjoy :)
This was more interesting/amusing than the OP!
Thank you. It seems like quite a few people liked it :)
Here’s the first write up with the CLI version: https://nicobrenner.github.io/lossy-git/
And repo code: https://github.com/nicobrenner/lossy-git
Dumb javascript is preventing me from seeing the footer, by scrolling back up to some content it thinks I shiould see.
Kudos. We did/built something internally to solve similar-ish problems https://www.shayon.dev/post/2026/283/versioned-filesystem-fo...
This website behaves very strangely... scroll hijack. Also everything is suuuuuper stretched out.
If you scroll down enough, you can't scroll up anymore!
EDIT: If you scroll down to the commit graph and have your mouse inside the big commit graph rectangle, you can't scroll up anymore!
ShowHN: WoolJ.ai the virtual wool jersey tailor-made for your most special coding agents
Folks might also be interested in try [1]. Lets you run anything (including your agent) in a mirror of your current system and then inspect/apply/discard/++ any of its effects.
[1]: https://github.com/binpash/try
How is it v1 with an architecture page coming soon? I'm really having a hard time gauging the maturity of software & services these days.
The evergreen observation; The most self-promote-y among us are mostly using AI to reinvent solutions and disguise them as new designs. These address solved problems, which are themselves disguised as new agentic problems.
Celebrating and promoting this stuff does not help us get better at discovering and building actually new things; change my mind.
UnionFS and overlay2, and the concept of layer filesystems existing makes this naming unfortunate and confusing.
We went from copy/paste this | sudo bash --> Copy this large prompt that installs stuff | AI Agent
How is this any different than something like btrfs?
It is more content aware and tuned towards versioned data vs block aware among other things. There’s a benchmark as to how it impacts things in the site
https://layerfs.ai/benchmarks/dsh-dedup/
I think it’s actually a fairly useful thing but I worry about something so obviously vibe coded hitting 1.0.0 and without a track record but the ideas seem pretty useful. I’ve certainly found the work tree semantics most agents and harnesses follow at scale to be obnoxious at best and it seems to tackle issues head on.
Nice benchmarks, impressive numbers. 20x less space than btrfs for the deep seek harness.
They even have some explanations. Files writes in btrfs snapshots recreatethe contents as opposed to storing what's changed. Btrfs has no dedupe, if multiple snapshots happen to have the same data. And small files store metadata twice in btrfs for reasons.
I do think their use case is a bit pathological, checking out the same harness at every commit. Most of these btrfs gotchas don't feel like they would be super impactful to me. But really cool results none the less.
I have this pattern but because I run pipelined agents working on a speculative merge topologically sorted task list merging into a merge train. This leads to lots and lots of work trees that largely overlap except for slight variants. Their articulated use case is mine, and I can have up to a hundred work trees at a time. The storage and related back pressures can be complicated.
For the source code part of that, the difference between using layerfs and btrfs snapshots feels near trivial. There'd be so little difference at the source layer. Unless your changes are massive massive massive repo wide changes, most of the content is going to remain the same, and the btrfs snapshot is going to mean each workspace is a proportionally a tiny delta, have as many as you want.
I guess the win is that often build artifacts are the same in many cases, so a pristine checkout still de-dupes. There's very good build caching systems out there that help with not needing to keep re-compiling the exact same stuff you've compiled, so I tend to discredit this usage some. But each of those workspaces would end up with it's own copy of the build artifacts, so there is a real savings to layerfs in either case, with it's dedupe.
Good for you on your "speculative merge topologically sorted task list merging into a merge train". Fun hard problems to have! Great. I don't want to be too rude, but also, personally, I think you could have simmered that down a bit, at the cost of not presenting quite entirely how super advanced elite super cool you are.
Perhaps but it wouldn’t have been explicit about why I have that problem. I don’t think it’s a normal problem to have, whether it’s elite or not, it’s IMO not common. The speculative pipeline and the merge train is the key part because they overlap with changes in the same files. This is why block based doesn’t help me as much. I’d prefer to not censor myself with respect to my problem statements for some internet humility virtue signaling.
"I have lots of builds on the same tree" is not really losing much relevant signal. I don't think I'm trying to make you earn "virtue signaling" humility points, I'm trying to increase the chance people understand and ascertain what you're actually saying. Without having to spend time digging through all the inflated noise you're giving off. Communication and brevity are worthy objectives.
If your commitment is just to bragging keep doing what you're doing. I'm just trying to help you be effective. Apologies, I guess? You seem not only not interested, but you're showing some gross colors in the process (virtue is good, actually).
As much as I appreciate your intention might be to be helpful it was in fact quite rude and arrogant. I think telling someone in a technical forum to not be so technical and lecturing them on clarity of communication is pretty offensive - you don’t know me, know what I do or who I am, what my background is, and clearly my message wasn’t intended for your consumption since you found it not for your consumption. Some advice since we are dishing it out, this sort of stuff is as annoying as correcting grammar on Usenet posts or all the other forms of things that disingenuously begin with “not to be rude” - if you are aware it is likely to be read as rude, maybe just not type it next time? Just trying to help you be effective and not show such gross colors next time.
PS, it’s about builds on the same tree but about edits on the same set of files in the tree that would cause block COWs in brtfs. If you read the original links docs you will see that’s the use case they’re targeting, and it’s a lot different than build artifacts. Their proposal isn’t just assisting in deduping but in merging rapidly files that have interleaved changes.
I started on a bad foot when you messaged me, and here's why it rubbed me wrong: it seemed like btrfs snapshots would give you 98% of the wins just as easily. With far better reliability than using a nosync sqlite as a backing store. And I'd talked thought some of the differences. But it didn't seem like you were connecting with or going further into any of the material or tradeoffs or how the two would meet your workload. You didn't connect your work to the subject at all other than to show up and plant a flag. I found it rude and arrogant that you'd just show up to share your thing and be so technically disconnected from the topic.
That all made me pretty sore. Complaining about virtue signalling & this post here, well, we're done here. I don't feel like you've done anything to connect me with the problems. Btrfs seems like it would work just fine for your use cases. You haven't talked to the problem or drawn any connection. Not really interested anymore. Good luck.
Or this https://arxiv.org/html/2602.08199v1
The difference is this one (layerfs) was vibe coded™ and the other (btrfs) is battle tested in mission critical systems for years in the Linux kernel and used by many companies in the SNP 500.
Do not be fooled by the "vibes" of the website design which many here are. If it is too good to be true, it most likely is.
"btrfs is battle tested"
Only in a sense that i went to a battle and lost all of its data to corruption. Should be removed from in-tree so people don't accidently use it.
That's what I was concerned about. An unstable filesystem is worse than no filesystem at all; I'd really like to switch to bcachefs but it simply needs to have a few more years under its belt before I'll be comfortable with it (particularly for some of its best use cases, where it becomes responsible for backups).
I would not trust btrfs with anything mission critical.
It's fine if you stay away from it's raid5/6. And realize it isn't going to let you know when you have a mirror fail until you try to boot it and it won't come up.
> is battle tested in mission critical systems for years
And still crashes and corrupts filesystems all the time.
(I have one server platform where just doing `insmod btrfs` immediately locks up the server, hard, without even any kernel panic or log message.)
How recent is that code?
Less than 12 months apparently.
Nice website, but that's about it
> Nice website
I disagree. Its full of AI slop. Too much animation (cute but distracting) I literally tried for 20 minutes and couldn't focus on reading. I just couldnt comprehend anything.
I'm so glad someone else couldn't focus I thought I was losing my mind here.
ho yes please hack my browser scroll so I can't read your site (MacOS Safari)
~245k lines of Rust the LLMs coded. So my question is other than DSH does the author(s) have any advice on setup that made creating this possible? or other have tips for best setup that can produce a project like this.
Even those bashing for being AI generated have to admit it’s technically impressive and reflective of where software engineering is headed.
I have to try it but I am very well interested in the decentralized agents mentioned there, I am tired of scattered agents and sessions across multiple OSes and harnesses I have, I just need one say an LXC that I can self host and automatically sync (or rather async like push/pull) all my harnesses, regardless wherever the previous session was.
You can achieve this by externalizing the memory outside of every harness and then connecting all harnesses to it via MCP. Then you will have a single store where you can both read and write, and nothing will need to be synced.
I am the founder of Substrate, a project that aims to provide this capability: having just one memory for all your agents, sessions and tools.
What two harnesses do you switch between most often?
Not memory, rather the sessions and skills. So you start your session in opencode on linux with say sol model, then the next hour you pull your Mbp start claude code on macos and cont the session on opus, with the same skills too. Memory isn’t that useful I found it actually bad as it impacts the output in a different direction than what was asked. I was thinking to build it but didn’t have the time, if something proven exists and open source (to trust it with sessions) then that would be great.
If the skill wa already loaded into context tranplanting it into a new harness will move that skills context as well (not the scripts though).
What kind of memory has degraded your performance? Memory is quite tricky to tune and simple a direct implementations of an LLM Wiki concept can indeed have a lot of problems, as context accumulates.
> start your session in ... then ... cont the session on
This is not a technical restriction, but a commercial one. Any vendor agnostic harness can implement session sharing as a feature, but because you would be made to pay higher token pricing, there will be little interest among the consumers.
You can do session sharing in proprietary harnesses as well - we do that at Substrate. Sessions are just text - and can be manipulated however you want. lmk if you want a quick onboarding to trysubstrate.co
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[dead]