> 2026-03-02: Ruohang Feng has forked MinIO to pgsty/minio and is promising to maintain a stable, CVE-patched, distribution.
This is what I went with.
I use it in end to end tests as an S3 simulator that starts and stops instantly and reads & writes to a local directory, - as you'd expect it's great in that role. No complaints. Given the fork's maintainer puts their real name on it & stakes their reputation, you'd assume it can be trusted - but my use case is simpler than most.
I had switched some projects to SeaweedFS's Docker image. It worked well. This is good to know there is a successor open source project that would require no code changes to use. Will evaluate.
This is what I use. I had a minio instance that was replaced flawlessly with silo. This is an excellent project and needs more traction. All existing minio users must switch to Silo ASAP.
It's nice to hear about other products, but OP made it sound like Minio disappeared and was not an option. Neither are true. Yes the company moved on but the old source is sitting there and that version is pretty good as-is. We're still using it in prod, no issues for us. The old container images are floating around and you're free to copy them to wherever.
If you need support or newer features, fork it or use ^ this fork.
I use that aswell since it can be used and expose a posix filesystem. This is helpful when we want to add s3 to existing storage like a nas. (Well we had some problems with it in the past tough)
Sadly most new projects do not offer posix anymore.
I've been using it for local development for a while now. My goal was to replace MinIO, which I could run and configure with a single Docker command. VersityGW works just as well for that.
> It's also nice that it translates directly to the file system.
On the other hand, this means that it's bound by filesystem naming rules - e.g. you can't have an object named "abc/", or a pair of objects named "a" and "a/b".
Running Sentry on prem, I had to swap out their default SeaweedFS setup for Garage because the former kept failing under concurrent writes. I tried digging around for a bit, but found that the swap was easier and faster, Garage has also worked great for single node use cases (e.g. tested up to around 10 TB of data). I still think that SeaweedFS is a cool project, might have been a config issue or something, wasn’t worth tweaking.
The setup for Garage sucks, especially cause their Docker image doesn’t automatically create keys or buckets and permissions for you like for example various RDBMS images do. Doing that the first time manually was annoying, but their docs are pretty nice and an AI agent can build you your own Docker image with custom init in about 15 minutes.
Worst with it I’ve had were issues with hooking up WinSCP to it directly to browse saved satellite data, initial connections would hang for some reason, not sure what the problem was either.
Also used Zenko but kinda got the feeling that the project wasn’t as healthy and straight up felt abandoned (e.g. the outdated container images and such), though there is some activity.
SeaweedFS performance should be the fastest. Please create an issue if there are any performance issues with SeaweedFS, or you see it is slower than any alternatives.
Not without snapshot currently. I guess I could collect all writes in the (optional) WAL file while a backup is running, then the files would be consistent.
The title is 'for single node local S3' but what they actually mean is 'for minio-compatibility', which is an entirely different question.
When I abandoned minio a year or so ago, I also surveyed the options, and settled on Garage. It lacked the GUI, but felt about the same complexity as minio. Perhaps a smidge more complexity, as I moved to 3-node and 5-node separate instances of garage, running as containers under Nomad.
I don't recall it being onerous, but I was looking for some basic S3-alike capabilities, not just minio-alike.
(How many people set out to build an object storage system with some number of AWS S3 primitives, but primarily try to match a third-party proprietary system's foibles?)
> So, Garage does work, but gosh…it is not just a drop-in replacement in terms of code changes.
I think in terms of actual code that uses local S3, it pretty much was a drop-in replacement. (I have multiple distribution/registry, Grafana Loki / Mimir, influx3 - all backing onto my object storage system, and the config changes there were modest - key+secret, and url - just as you'd expect.)
Filestash (https://github.com/mickael-kerjean/filestash) has a s3 gateway plugin that I made. It proxy the S3 traffic to any downstream storage: SFTP, FTP, another S3, SMB, NFS, IPFS, ...
How does that handle concurrency (e.g. updates being atomic, conditional PUT) and data consistency after a crash? Getting that right over generic backends sounds challenging, to say the least.
Almost went past looking at this option, it has a far bigger ecosystem of both storage and auth plugins and otherwise looks to be more mature than I might have guessed from the passing comment.
Was the backend for an archiving service and we found it to work well on baremetal machines and seamlessly integrated with existing services that used the S3 api.
This was on an internal network so YMMV when it comes to running on the open internet especially with the CVE mentioned in the post.
I had previously used seaweedfs, which has more moving parts but seems more feature-rich. It was overkill for a homelab, so I went to garage.
Garage is very stable, but the metadata engine / file MUST live on NVMe. Otherwise it gets slow on small files. It also lacks features like per bucket retention settings and stuff. It really is "just" a S3 bucket.
I wouldnt use Garage for Iceberg, Iceberg relies on conditional PUT working correctly, while Garage is eventually consistent, so they don't provide the correct guarantee.
Is there any "local"/minio-esque object store properly implements the full conditional GET/PUT/DELETE spec, ETag, etc.? I have struggled to find this in the past.
Awesome, I will take a look! We need something for our end-to-end tests at work that we can use and trivially spin up buckets for millions of tests in parallel.
Does anyone here have experience using some of the newer S3 family API features like locking with any of the self-hostable options? After reading about celld's Litestream + locking on S3 approach (https://github.com/denoland/celld) I got curious about that style of deployment as a way to consolidate small tools so their state is just centralize S3 buckets, but from a bit of reading support for the features like locking is uneven.
Its not locking, it needs consistent conditional PUT. Minio did have that, Rustfs does, Garage is eventually consistent so explicitly does not support it correctly.
I did try seaweedfs ~half a year ago and I had issues with setting up users through terraform module (if I recall correctly, users endpoint were not correctly responding on delete), and lack of S3 expiry rules (you needed to use seaweedfs configuration or API for that). Other than that, I was pretty happy with it.
- Filesystem backend
- Encrypted S3 backend
- Compressed S3 backend (Saving a ton of money here)
- Cross backend replication
- Full IAM
- Full GUI with OIDC auth
I run Scality S3 (Zenko Cloudserver in the blig) since about 2 years and it is quite capable and frequently updated. Just make sure to point it to the latest Github Registry, not the outdated Docker.com one.
Yup, same. We use RustFS in a production system with no issues. We switched over after MinIO license changes and after evaluating a few different stacks.
We have heavy concurrent usage, and we haven't had a single issue yet.
I ended up just getting Claude to hack together a compatible-enough S3 thing for my self-hosted Sourcehut releases. It took like 45 minutes and it seems to work well enough (though I haven't done exhaustive tests because "good enough is good enough" for this).
The S3 API is pretty well-documented so getting something compatible with it is trivial for modern AI tools.
I'm still disappointed, that there is no S3 CNCF sponsored process. It seems to be such an important part of the cloud native ecosystem, every big use case needs some way to store blobs of data.
But no, I'm also not about to start one or try to make something like SeaweedFS a community supported project that has a chance to be picked up by the CNCF, so I'm part of the problem.
Ceph is pretty much the standard, I think. Very complex to manage, though, but that might just be necessary complexity for a distributed storage system.
The recent RustFS CVE has driven some people away from RustFS, based on comments from the community.
For those who had similar past experiences:
Do events like this actually affect the long-term popularity and development?
Ethnicity never mattered (other than for the people that think too many white people are doing something). It's culture, in particular government ties, that are the worry with China.
In the decades that open-source projects have existed on GitHub—even those backed by the Chinese government—there has never been an instance of them containing spyware; conversely, we frequently see NSA hacking tools exploiting open-source projects to launch massive cyberattacks and cause data breaches. Reality isn't like American movies where Americans are always the heroes; in fact, those GIs are usually the ones doing the invading. To me, your statement sounds like a resident living under Nazi rule telling everyone, "There's actually a pretty good project over on the Allied side—if you choose to believe it."
It's nice, I am just afraid of the pace Chris Lu can maintain with LLM assistance. Too many features are added in each new release, no stabilization time at all.
Because you want to mock the tools you are going to use in production locally or in CI. Really good reasons to not totally change what you are doing in your app between environments.
The same argument can be made for the other side: the customer can also trivially mount the S3 bucket as a filesystem, if you decide to ship the tool with filesystem support instead.
But is that really something you do in an enterprise application?
What if you write a file with the same name from two servers to the bucket? What about latency? What if the connection / mount fails, how would the application even notice?
I've dabbled in both Garage and RustFS. Both are good but IMO flawed.
Garage is good but has weird tooling and a terrible searchable term. You'll receive lots of garage doors and electrical gate results when searching for issues with it.
The biggest problem with Garage is the metadata on an sqlite. If you have a single file and that sqlite corrupts then all your data is unaddressable and treated as pile of anonymous blocks. If you have multiple servers you can rebuild the metadata but if you have only one server you REALLY need to make sure that file is backed up.
So if you use S3 as a backup you have to keep a backup of your backup.
RustFS is closer to Minio, but I hate the way they log stuff. They basically use journald as a metric database dumping json formatted lines about library calls and how long they took. Its hard to make sense of those logs. Its the kind of log not meant to be human readable but to render on a dashboard.
Why not just publish these as a prometheus endpoint?
Also high CPU usage for no reason. I also noticed RustFS constantly trashing disk and keeping 10% CPU usage while looping some health check I have no idea how to disable (I've tried RUSTFS_SCANNER_ENABLED=false and others with no results). And of course lots of json metrics on journald.
I personally prefer garage because my backup box sit at my office and I like not needing to listen to HDD sounds all day every day while Rustfs loops its health check.
> 2026-03-02: Ruohang Feng has forked MinIO to pgsty/minio and is promising to maintain a stable, CVE-patched, distribution.
This is what I went with.
I use it in end to end tests as an S3 simulator that starts and stops instantly and reads & writes to a local directory, - as you'd expect it's great in that role. No complaints. Given the fork's maintainer puts their real name on it & stakes their reputation, you'd assume it can be trusted - but my use case is simpler than most.
According to https://github.com/pgsty/silo it's pgsty/silo now:
> Renamed from pgsty/minio to pgsty/silo, default branch master → main, on 2026-08-06
This need to go mainstream.
Neat!
I had switched some projects to SeaweedFS's Docker image. It worked well. This is good to know there is a successor open source project that would require no code changes to use. Will evaluate.
This is what I use. I had a minio instance that was replaced flawlessly with silo. This is an excellent project and needs more traction. All existing minio users must switch to Silo ASAP.
Right.
It's nice to hear about other products, but OP made it sound like Minio disappeared and was not an option. Neither are true. Yes the company moved on but the old source is sitting there and that version is pretty good as-is. We're still using it in prod, no issues for us. The old container images are floating around and you're free to copy them to wherever.
If you need support or newer features, fork it or use ^ this fork.
I might have to try this. I’m currently pinned on the last non-crippled version of minio
Garage added an automatic configuration feature in v2.3.0 that makes it easier to set up single nodes:
https://garagehq.deuxfleurs.fr/documentation/quick-start/I can vouch for Garage too. It was a lot simpler imo than minio and works great over a handful of servers in different DCs.
A notable mention should also be Versity GW, https://github.com/versity/versitygw/
+1 switched to versity for homelab, been simple and stable as a rock. Point at a directory per bucket and done.
Also discussions few months ago here when healthcheks.io switched to it: https://news.ycombinator.com/item?id=47806348
I use that aswell since it can be used and expose a posix filesystem. This is helpful when we want to add s3 to existing storage like a nas. (Well we had some problems with it in the past tough)
Sadly most new projects do not offer posix anymore.
I've been using it for local development for a while now. My goal was to replace MinIO, which I could run and configure with a single Docker command. VersityGW works just as well for that.
This is what I switched to as well, works nicely. Can have just a folder as a backend as well, which makes things pretty simple.
Use it for my homelab too, super nice and stable. Support also delivering static website/asset.
https://github.com/versity/versitygw/#static-website-hosting
I'm quite pleased with it. It's also nice that it translates directly to the file system .
> It's also nice that it translates directly to the file system.
On the other hand, this means that it's bound by filesystem naming rules - e.g. you can't have an object named "abc/", or a pair of objects named "a" and "a/b".
I've had good experiences with Versity GW too and it's a shame it's not more well known.
Running Sentry on prem, I had to swap out their default SeaweedFS setup for Garage because the former kept failing under concurrent writes. I tried digging around for a bit, but found that the swap was easier and faster, Garage has also worked great for single node use cases (e.g. tested up to around 10 TB of data). I still think that SeaweedFS is a cool project, might have been a config issue or something, wasn’t worth tweaking.
The setup for Garage sucks, especially cause their Docker image doesn’t automatically create keys or buckets and permissions for you like for example various RDBMS images do. Doing that the first time manually was annoying, but their docs are pretty nice and an AI agent can build you your own Docker image with custom init in about 15 minutes.
Worst with it I’ve had were issues with hooking up WinSCP to it directly to browse saved satellite data, initial connections would hang for some reason, not sure what the problem was either.
Also used Zenko but kinda got the feeling that the project wasn’t as healthy and straight up felt abandoned (e.g. the outdated container images and such), though there is some activity.
SeaweedFS performance should be the fastest. Please create an issue if there are any performance issues with SeaweedFS, or you see it is slower than any alternatives.
(I work on SeaweedFS)
I've been watching seaweedfs for a few years now. It is very cool tech and I'm itching to use it but unfortunately the docs are very bare bones.
Do you have plans to improve the docs? Some of the info I needed was inside GitHub issues / discussions.
Overall very cool project. One of these days I'll reach out about enterprise support for my usecase.
I am not sure whether you checked seaweedfs github wiki. Many people did not notice the many wiki pages there.
My https://github.com/uroni/hs5 is designed for this use case.
One notable thing is that compared to MinIO (and others) it does not store the objects as individual files. I also have DuckDB directly integrated.
The readme has a comparison to Garage, seaweedfs, RustFS and Ceph.
Very cool, I like the conceptual simplicity of it. Even I can understand it, at least at a surface level :-)
Is there a safe way to make backups of the index.lmdb and data0 files without stopping the server?
Not without snapshot currently. I guess I could collect all writes in the (optional) WAL file while a backup is running, then the files would be consistent.
A weird target, the author has.
The title is 'for single node local S3' but what they actually mean is 'for minio-compatibility', which is an entirely different question.
When I abandoned minio a year or so ago, I also surveyed the options, and settled on Garage. It lacked the GUI, but felt about the same complexity as minio. Perhaps a smidge more complexity, as I moved to 3-node and 5-node separate instances of garage, running as containers under Nomad.
I don't recall it being onerous, but I was looking for some basic S3-alike capabilities, not just minio-alike.
(How many people set out to build an object storage system with some number of AWS S3 primitives, but primarily try to match a third-party proprietary system's foibles?)
> So, Garage does work, but gosh…it is not just a drop-in replacement in terms of code changes.
I think in terms of actual code that uses local S3, it pretty much was a drop-in replacement. (I have multiple distribution/registry, Grafana Loki / Mimir, influx3 - all backing onto my object storage system, and the config changes there were modest - key+secret, and url - just as you'd expect.)
Filestash (https://github.com/mickael-kerjean/filestash) has a s3 gateway plugin that I made. It proxy the S3 traffic to any downstream storage: SFTP, FTP, another S3, SMB, NFS, IPFS, ...
How does that handle concurrency (e.g. updates being atomic, conditional PUT) and data consistency after a crash? Getting that right over generic backends sounds challenging, to say the least.
Almost went past looking at this option, it has a far bigger ecosystem of both storage and auth plugins and otherwise looks to be more mature than I might have guessed from the passing comment.
We ran RustFS in production at a past job.
Was the backend for an archiving service and we found it to work well on baremetal machines and seamlessly integrated with existing services that used the S3 api.
This was on an internal network so YMMV when it comes to running on the open internet especially with the CVE mentioned in the post.
This has been my solution as well. It has recently been added as an integrated service via devenv:
https://devenv.sh/services/rustfs/
I had previously used seaweedfs, which has more moving parts but seems more feature-rich. It was overkill for a homelab, so I went to garage.
Garage is very stable, but the metadata engine / file MUST live on NVMe. Otherwise it gets slow on small files. It also lacks features like per bucket retention settings and stuff. It really is "just" a S3 bucket.
For testing/dev setups you can turn sync off and get some performance back but that's only useful for cases where losing data is okay.
The trouble I had with SeaweedFS was that the docs I found were rather sparse. Garage, on the other hand seemed pretty well documented.
Did you miss the wiki or was the wiki not enough? Please let me know. I work on SeaweedFS.
To be honest, I was going off memory of a couple years back, and having revisited the wiki it does seem to have a lot more information!!
The very first thing Garage did after I installed it on a test three-node cluster is get corrupted and lose files.
No thanks.
I wouldnt use Garage for Iceberg, Iceberg relies on conditional PUT working correctly, while Garage is eventually consistent, so they don't provide the correct guarantee.
Is there any "local"/minio-esque object store properly implements the full conditional GET/PUT/DELETE spec, ETag, etc.? I have struggled to find this in the past.
I implemented this (HS5). Etags match the AWS S3 ones as well.
Awesome, I will take a look! We need something for our end-to-end tests at work that we can use and trivially spin up buckets for millions of tests in parallel.
For single-node local S3, I often just use `s3fs` mounting a local directory. MinIO always felt like overkill for simple dev.
Does anyone here have experience using some of the newer S3 family API features like locking with any of the self-hostable options? After reading about celld's Litestream + locking on S3 approach (https://github.com/denoland/celld) I got curious about that style of deployment as a way to consolidate small tools so their state is just centralize S3 buckets, but from a bit of reading support for the features like locking is uneven.
Its not locking, it needs consistent conditional PUT. Minio did have that, Rustfs does, Garage is eventually consistent so explicitly does not support it correctly.
Ah! Thanks. For anyone else following this is documented along side the replication and failover behavior here:
https://celld.dev/docs/guarantees/
I did try seaweedfs ~half a year ago and I had issues with setting up users through terraform module (if I recall correctly, users endpoint were not correctly responding on delete), and lack of S3 expiry rules (you needed to use seaweedfs configuration or API for that). Other than that, I was pretty happy with it.
Thanks! I am sure most of the bugs are fixed now. SeaweedFS now should have the most complete S3 API support now.
(I work on SeaweedFS)
Really happy with my own https://deltaglider.com/
- Filesystem backend - Encrypted S3 backend - Compressed S3 backend (Saving a ton of money here) - Cross backend replication - Full IAM - Full GUI with OIDC auth
- slop code [0]
- slop copy [1]
[0] https://github.com/beshu-tech/deltaglider_proxy/graphs/contr...
[1] https://deltaglider.com/pricing/
Thanks for the feedback @Kwpolska!
I made it explicit Deltaglider was created by SWEs, aided by Fable 5.x.
Also, I asked Fable to un-slop the copy of the website, using this excellent skill https://github.com/blader/humanizer
Check it out, once again: https://deltaglider.com/
Lol. You made 1254 commits. 1179 have a Claude attribution, and 38 have a Cursor attribution. That leaves 37 commits written by you. This is 97% slop.
97% AI assisted, yes!
I actually wanted to proof a point to myself (or not!): is it possible to obtain quality results without almost touching code?
More than half of tokens was burned in quality gates, adversarial reviews, and shaping a solid QA pyramid. Took forever. But it did work out.
Been dogfooding it live in my company for a few months now. It's VERY stable, no errors. Security audit was ok too.
Do you have any feedback? Even the license is up to discussion!
Incus (spiritual successor to LXD, after the fork-off by Canonical) has a simple S3 server built-in.
Isn’t that just minio? At least in LXD it’s minio.
Replaced because minio is dead: https://github.com/lxc/incus/pull/3258
Nice!
I run Scality S3 (Zenko Cloudserver in the blig) since about 2 years and it is quite capable and frequently updated. Just make sure to point it to the latest Github Registry, not the outdated Docker.com one.
One alternative worth looking at is S2: https://github.com/mojatter/s2
Versity's posix backend was the one that stuck for us, multipart uploads land as real files on disk so you can inspect them with ls when a test fails.
Rustfs, works pretty well for me
Yup, same. We use RustFS in a production system with no issues. We switched over after MinIO license changes and after evaluating a few different stacks.
We have heavy concurrent usage, and we haven't had a single issue yet.
i thought rustfs was AI slop?
I ended up just getting Claude to hack together a compatible-enough S3 thing for my self-hosted Sourcehut releases. It took like 45 minutes and it seems to work well enough (though I haven't done exhaustive tests because "good enough is good enough" for this).
The S3 API is pretty well-documented so getting something compatible with it is trivial for modern AI tools.
Start a ready-to-use SeaweedFS S3 object store:
AWS_ACCESS_KEY_ID=admin AWS_SECRET_ACCESS_KEY=secret S3_BUCKET=my-bucket ./weed mini -dir=./data
That's it.
The S3 endpoint is at http://localhost:8333, my-bucket exists, and admin/secret are valid credentials.
It also starts an Iceberg REST catalog and S3 Table Buckets for Iceberg and Lance tables, with table maintenance jobs.
I'm still disappointed, that there is no S3 CNCF sponsored process. It seems to be such an important part of the cloud native ecosystem, every big use case needs some way to store blobs of data.
But no, I'm also not about to start one or try to make something like SeaweedFS a community supported project that has a chance to be picked up by the CNCF, so I'm part of the problem.
Ceph is pretty much the standard, I think. Very complex to manage, though, but that might just be necessary complexity for a distributed storage system.
(I work on SeaweedFS) Needs to pay to join CNCF, about $8k/year. I would like more sponsorship.
The recent RustFS CVE has driven some people away from RustFS, based on comments from the community. For those who had similar past experiences: Do events like this actually affect the long-term popularity and development?
I use JuiceFS https://github.com/juicedata/juicefs if you think the Chinese are worth the trust
If it's open source, you don't do daily pulls and you have a competent AI to crawl the codebase; why does it matter the ethnicity at this point?
Ethnicity never mattered (other than for the people that think too many white people are doing something). It's culture, in particular government ties, that are the worry with China.
Speaking as a Chinese myself and who contributed to JuiceFS before, yeah, the culture part, you're spot on...
Fair. I would say the same about American big tech's close ties with the American government, by the way.
In the decades that open-source projects have existed on GitHub—even those backed by the Chinese government—there has never been an instance of them containing spyware; conversely, we frequently see NSA hacking tools exploiting open-source projects to launch massive cyberattacks and cause data breaches. Reality isn't like American movies where Americans are always the heroes; in fact, those GIs are usually the ones doing the invading. To me, your statement sounds like a resident living under Nazi rule telling everyone, "There's actually a pretty good project over on the Allied side—if you choose to believe it."
https://www.npr.org/2024/02/22/1233178131/leaked-document-tr...
I enjoyed the smooth transition from minio to RustFS quite a lot.
I didn't. Hit bug after bug after bug in RustFS. Current version seems to work fine for now but it's definitely "new project"
On flipside the project maintainers react very fast on any bug I submitted and it's fixed pretty quickly, so not really complaint, just warning
I am using Garage for my local DEV environment.
https://garagehq.deuxfleurs.fr/
I don't think I know anyone running minio anymore. It's sad how they destroyed themselves, but that's the way it goes.
I'm pretty happy with Seaweed, running that on prem, rock solid so far. Moved off MinIO when the licensing shenanigans started
Often just spinning up `localstack` works well, even if only for S3. Gives you other AWS services if you ever need them.
Commercial use requires a paid subscription. Probably not an issue for a home lab, unless you run a small business on it.
Are any of these truly alternatives? MinIO is built for scale and not small projects. Ceph is maybe closest.
Define scale.
When I tried minio 5 years ago it choked pretty quick on storing billions of objects
as per the title the scope here is single-node local S3
I heard good news about SeaweedFS
It's nice, I am just afraid of the pace Chris Lu can maintain with LLM assistance. Too many features are added in each new release, no stabilization time at all.
Right. It takes full time to work on this. But I got help from a co-founder now.
Actually, AI does help a lot to setup all variations of testing scenarios and uncover edge case bugs. It's more helpful than vibes.
And you do need to be systematic to examine AI generated code. There are no shortcut for it.
or rclone serve
Came to say this.
If we're just talking verifying S3 connectivity, simply spinning up "rclone serve s3" appears to be the simplest solution.
SeaweedFS has a great IAM/STS/policy layer as well. It can support a secretless auth flow similar enough to AWS IRSA with it's OIDC integration
my pick would be s3proxy, been using it for a while, works really well in local dev
Til there are like 50 s3 clones
RustFS is great. Just works.
Just use the damn file system. Why does everyone have to put HTTP between everything?
Because you want to mock the tools you are going to use in production locally or in CI. Really good reasons to not totally change what you are doing in your app between environments.
> Really good reasons to not totally change what you are doing in your app between environments.
Mocks do that.
If you want to not totally change what you are doing in your app between environments, point it at an actual S3 bucket in all environments.
Because this is about having a local standin for S3, which (in a production deploy) often solves a different problem than a (local) filesystem.
This post does not even mention the S3 API as a replacement for a filesystem.
Because you might be running something that only talks S3 and want to point it at something local.
I like how you got 4 different comments all starting with "Because..." hah!
I can pay Amazon pennies on the dollar to saturate 100Gbps with arbitrarily high concurrency
Sometimes I also want a smaller version for when I don't need that performance but don't want multiple implementations of storage backends
Edit: before someone says NVMe, highly available across 3 fault domains
Because AFAIK the filesystem is at the same time a huge API and designed for a different use case. I can think of the following examples:
- very flat structures: storing hundred of thousands of files in a single directory will fail
- designed for local disks and NFS is a leaky abstraction:
probably many more reasonsMy point is to push the storage abstraction up a level. Then plug S3 or file system into it.
I wouldn’t say this if I hadn’t done this on the PB scale. On a file system! With billions of files!
People are blind to anything other than shitty cloud APIs now. It’s depressing.
Because if you ship a tool with s3 support, the customer can decide if he wants to store in the cloud, or spin up a docker container locally.
The same argument can be made for the other side: the customer can also trivially mount the S3 bucket as a filesystem, if you decide to ship the tool with filesystem support instead.
I did not know you could do that.
But is that really something you do in an enterprise application?
What if you write a file with the same name from two servers to the bucket? What about latency? What if the connection / mount fails, how would the application even notice?
Never tried it, I don't think it fits our needs.
Crazy how large of a vacuum MinIO left lol
[flagged]
[flagged]
[flagged]
Shout-out to rustfs.
I've dabbled in both Garage and RustFS. Both are good but IMO flawed.
Garage is good but has weird tooling and a terrible searchable term. You'll receive lots of garage doors and electrical gate results when searching for issues with it.
The biggest problem with Garage is the metadata on an sqlite. If you have a single file and that sqlite corrupts then all your data is unaddressable and treated as pile of anonymous blocks. If you have multiple servers you can rebuild the metadata but if you have only one server you REALLY need to make sure that file is backed up.
So if you use S3 as a backup you have to keep a backup of your backup.
RustFS is closer to Minio, but I hate the way they log stuff. They basically use journald as a metric database dumping json formatted lines about library calls and how long they took. Its hard to make sense of those logs. Its the kind of log not meant to be human readable but to render on a dashboard.
Why not just publish these as a prometheus endpoint?
Also high CPU usage for no reason. I also noticed RustFS constantly trashing disk and keeping 10% CPU usage while looping some health check I have no idea how to disable (I've tried RUSTFS_SCANNER_ENABLED=false and others with no results). And of course lots of json metrics on journald.
I personally prefer garage because my backup box sit at my office and I like not needing to listen to HDD sounds all day every day while Rustfs loops its health check.