Fil-C is one of the most underrated projects I've ever seen. All this "rewrite it in rust for safety" just sounds stupid when you can compile your C program completely memory safe.
So, a few things, some of which others have touched on:
1. Fil-C is slower and bigger. Noticeably so. If you were OK with slower and bigger then the rewrite you should have considered wasn't to Rust in the last ten years but to Java or C# much earlier. That doesn't invalidate Fil'C's existence, but I want to point that out.
2. You're still writing C. If the program is finished or just occasionally doing a little bit of maintenance that's fine. I wrote C for most of my career, it's not a miserable language, and you are avoiding a rewrite. But if you're writing much new code Rust is just so much nicer. I stopped writing any C when I learned Rust.
3. This is runtime safety and you might need more. Rust gives you a bit more, often you can express at compile time things Fil-C would only have checked at runtime, but you might need everything and languages like WUFFS deliver that. WUFFS doesn't have runtime checks. It has proved to its satisfaction during compilation that your code is safe, so it can be executed at runtime in absolute safety. Your code might be wrong. Maybe your WUFFS GIF flipper actually makes frog GIFs purple instead of flipping them. But it can't crash, or execute x86 machine code hidden in the GIF, or whatever, that's the whole point.
The issue with Fil-C is that it's runtime memory safety. You can still write memory-unsafe code, just now it is guaranteed to crash rather than being a potential vulnerability.
Guaranteed memory safety at compile time is clearly the better approach when you care about programs that are both functionally correct and memory safe. If I'm writing something that takes untrusted user input like a web API memory safety issues still end up as denial-of-service vulns. That's better, but it's still not great.
Not to disparage the Fil-C work, but the runtime approach has limitations.
I don't think that's a useful way of thinking about memory-safety - a C compiler that compiles any C program to `main { exit(-1); }` is completely memory-safe. It's easy to design a memory-safe language/compiler, the question is what compromises are being made to achieve it.
Other languages have runtime exceptions on out-of-bounds access, Fil-C has unrecoverable crashes. This makes it pretty unsuitable to a lot of use cases. In Go or Java (arbitrary examples) I can write a web service full of unsafe out-of-bounds array reads, any exception/panic raised is scoped to the specific malformed request and doesn't affect the overall process. A design that's impossible in Fil-C.
Then you run into the problem of infinite loops, which nothing can prevent (sans `main { exit(-1); }` or other forms of losing turing-completeness), and are worse than crashes - at least on crashes you can quickly restart the program (something something erlang).
try-catch isn't a particularly complete solution either if you have any code outside of it (at the very least, the catch arm) or if data can get preserved across iterations that can easily get messed up if left half-updated (say, caches, poisoned mutexes, stuck-borrowed refcells) so you'll likely want a full restart to work well too, and might even prefer it sometimes.
I don't think runtime error handling is impossible in Fil-C, at least in theory. But the use cases for that are fairly limited. Most errors like this are not anticipated, and if you did encounter them then there's little or nothing you can do useful in response. Furthermore, runtime handling to continue means code changes, thus coupling to the runtime environment. All of these things are bad. It is usually acceptable to fail fast and restart, or at least report the error.
I could have made Fil-C’s panic be a C++ exception if I had thought that it was a good idea. And then you could catch it if that’s what tickled your pickle
I just don’t like that design. It’s a matter of taste
Why are you talking like this is black and white? Many things being compile time checkable is better than no things being compile time checkable. The existence of some thing in rust that can only be checked at runtime does not somehow make all the compile time checks that are possible irrelevant.
(Also I think the commenter you're replying to just worded their comment innacurately, code that crashes instead of violating memory safety is memory safe, a compilation error would just have been more useful than a runtime crash in most cases)
There are several ways to safely provide array bounds check hints to the Rust compiler, in-fact there's a whole cookbook. But for many cases, yep, runtime check.
It's an apple to non-existent-apple comparison. Fil-C can't handle it even with extra code because Fil-C provides no recovery mechanism.
I also don't think it's that niche a use case. It's one encountered by every web server or web client (scope exception to single connection/request). Or anything involving batch processing, something like "extract the text from these 10k PDFs on disk".
Sure, it's not implemented in Fil-C because it is very new and the point of it is to improve things without extensive rewrites.
Generally, I think one could want to recover from errors. But error recovery is something that needs to be designed in. You probably don't want to catch all errors, even in a loop handling requests for an application. If your application isn't designed to handle the same kinds of memory access issues as we're talking about here, the whole thing turns into non-existent-apples to non-existent-apples lol.
.get() will bounds check and the compiler will optimize that away if it can prove safety at compile time. That leaves you 3 options made available in Rust:
- Explicitly unsafe
- Runtime crash
- Runtime crash w/ compile time avoidence when possible
Rust also has run-time crash checks in the form of run-time array bounds checks that panic. So let us not pretend that Rust strictly catches everything at compile-time.
It’s true that, assuming all things equal, compile-time checks are better than run-time. I love Rust. But Rust is only practical for a subset of correct programs. Rust is terrible for things like games where Rust simply can not prove at compile-time that usage is correct. And inability to prove correctness does NOT imply incorrectness.
I love Rust. I use it as much as I can. But it’s not the one true solution to all things.
Not trying to be a Rust advocate and I actually don't work in it personally.
But Rust provides both checked alternatives to indexed reads/writes (compile time safe returning Option<_>), and an exception recovery mechanism for out-of-bounds unsafe read/write. Fil-C only has one choice which is "crash immediately".
Minor nitpick. Or confusion on my part. In the filc_malloc function the call to calloc doesn't seem to allocate enough memory to store an AllocationRecord for each location in visible_bytes. Should it be:
There are many C++ programmers and we are not the same!
My original foray into GCs was making real time ones, and the Fil-C GC is based on that work. I haven’t fully made it real time friendly (the few locks it has aren’t RT-friendly) but if I had more time I could make it give you hard guarantees.
It’s already full concurrent and on the fly, so it won’t pause you
Fil-C has two major downsides: it slows programs down and it doesn't interoperate with non-Fil-C code, not even libc. That second problem complicates using it on systems other than Linux (even BSDs and macOS) and integrating it with other safe languages.
I would never say it's impossible, and you've done some amazing work, but I do wonder if the second problem is feasibly surmountable. Setting aside cross-language interop, BYOlibc is not really tolerated on most systems. Linux is fairly unique here with its strongly compatible syscall ABI.
You're right that it's challenging. I don't think it's infeasible.
Here's why:
1. For the first year of Fil-C development, I was doing it on a Mac, and it worked fine. I had lots of stuff running. No GUI in that version, though.
2. You could give Fil-C an FFI to Yolo-C. It would look sort of like the FFIs that Java, Python, or Ruby do. So, it would be a bit annoying to bridge to native APIs, but not infeasible. I've chosen not to give Fil-C such an FFI (except a very limited FFI to assembly for constant time crypto) because I wanted to force myself to port the underlying libraries to Fil-C.
3. Apple could do a Fil-C build of their userland, and MS could do a Fil-C build of their userland. Not saying they will do it. But the feasibility of this is "just" a matter of certain humans making choices, not anything technical.
I think there's two main avenues for hardware acceleration: pointer provenance and garbage collection. The first dovetails with things like CHERI [1] but the second doesn't seem to be getting much hardware attention lately. It has been decades since Lisp Machines were made, and I'm not aware of too many other architectures with hardware-level GC support. There are more efficient ways to use the existing hardware for GC though, as e.g. Go has experimented with recently [2].
It makes more sense for new software to be written in Rust, rather than a full rewrite of existing C/C++ software to Rust in the same codebase.
Fil-C just does the job with existing software in C or C++ without an expensive and bug riddled re-write and serves as a quick protection layer against the common memory corruption bugs found in those languages.
Fil-C is much slower, no free lunch, if you want the language to be fast and memory safe you need to add restrictions to allow proper static analysis of the code.
This is yet another variant of the "fat pointers" technique, which has been implemented and rejected many times due to either insufficient security guarantees, inability to cross non-fat ABI boundaries, or the overhead it introduces.
Fil-C is one of the most underrated projects I've ever seen. All this "rewrite it in rust for safety" just sounds stupid when you can compile your C program completely memory safe.
So, a few things, some of which others have touched on:
1. Fil-C is slower and bigger. Noticeably so. If you were OK with slower and bigger then the rewrite you should have considered wasn't to Rust in the last ten years but to Java or C# much earlier. That doesn't invalidate Fil'C's existence, but I want to point that out.
2. You're still writing C. If the program is finished or just occasionally doing a little bit of maintenance that's fine. I wrote C for most of my career, it's not a miserable language, and you are avoiding a rewrite. But if you're writing much new code Rust is just so much nicer. I stopped writing any C when I learned Rust.
3. This is runtime safety and you might need more. Rust gives you a bit more, often you can express at compile time things Fil-C would only have checked at runtime, but you might need everything and languages like WUFFS deliver that. WUFFS doesn't have runtime checks. It has proved to its satisfaction during compilation that your code is safe, so it can be executed at runtime in absolute safety. Your code might be wrong. Maybe your WUFFS GIF flipper actually makes frog GIFs purple instead of flipping them. But it can't crash, or execute x86 machine code hidden in the GIF, or whatever, that's the whole point.
Not here, lots of discussion:
Fil-Qt: A Qt Base build with Fil-C experience (143 points, 3 months ago, 134 comments) https://news.ycombinator.com/item?id=46646080
Linux Sandboxes and Fil-C (343 points, 4 months ago, 156 comments) https://news.ycombinator.com/item?id=46259064
Ported freetype, fontconfig, harfbuzz, and graphite to Fil-C (67 points, 5 months ago, 56 comments) https://news.ycombinator.com/item?id=46090009
A Note on Fil-C (241 points, 5 months ago, 210 comments) https://news.ycombinator.com/item?id=45842494
Notes by djb on using Fil-C (365 points, 6 months ago, 246 comments) https://news.ycombinator.com/item?id=45788040
Fil-C: A memory-safe C implementation (283 points, 6 months ago, 135 comments) https://news.ycombinator.com/item?id=45735877
Fil's Unbelievable Garbage Collector (603 points, 7 months ago, 281 comments) https://news.ycombinator.com/item?id=45133938
The issue with Fil-C is that it's runtime memory safety. You can still write memory-unsafe code, just now it is guaranteed to crash rather than being a potential vulnerability.
Guaranteed memory safety at compile time is clearly the better approach when you care about programs that are both functionally correct and memory safe. If I'm writing something that takes untrusted user input like a web API memory safety issues still end up as denial-of-service vulns. That's better, but it's still not great.
Not to disparage the Fil-C work, but the runtime approach has limitations.
> write memory-unsafe code, just now it is guaranteed to crash
If it's guaranteed to crash, then it's memory-safe.
If you dislike that definition, then no mainstream language is memory-safe, since they all use crashes to handle out of bounds array accesses
I don't think that's a useful way of thinking about memory-safety - a C compiler that compiles any C program to `main { exit(-1); }` is completely memory-safe. It's easy to design a memory-safe language/compiler, the question is what compromises are being made to achieve it.
Other languages have runtime exceptions on out-of-bounds access, Fil-C has unrecoverable crashes. This makes it pretty unsuitable to a lot of use cases. In Go or Java (arbitrary examples) I can write a web service full of unsafe out-of-bounds array reads, any exception/panic raised is scoped to the specific malformed request and doesn't affect the overall process. A design that's impossible in Fil-C.
Then you run into the problem of infinite loops, which nothing can prevent (sans `main { exit(-1); }` or other forms of losing turing-completeness), and are worse than crashes - at least on crashes you can quickly restart the program (something something erlang).
try-catch isn't a particularly complete solution either if you have any code outside of it (at the very least, the catch arm) or if data can get preserved across iterations that can easily get messed up if left half-updated (say, caches, poisoned mutexes, stuck-borrowed refcells) so you'll likely want a full restart to work well too, and might even prefer it sometimes.
I don't think runtime error handling is impossible in Fil-C, at least in theory. But the use cases for that are fairly limited. Most errors like this are not anticipated, and if you did encounter them then there's little or nothing you can do useful in response. Furthermore, runtime handling to continue means code changes, thus coupling to the runtime environment. All of these things are bad. It is usually acceptable to fail fast and restart, or at least report the error.
I could have made Fil-C’s panic be a C++ exception if I had thought that it was a good idea. And then you could catch it if that’s what tickled your pickle
I just don’t like that design. It’s a matter of taste
By that token, Rust is also memory unsafe: array bounds checks and stack overflow are runtime checks.
Why are you talking like this is black and white? Many things being compile time checkable is better than no things being compile time checkable. The existence of some thing in rust that can only be checked at runtime does not somehow make all the compile time checks that are possible irrelevant.
(Also I think the commenter you're replying to just worded their comment innacurately, code that crashes instead of violating memory safety is memory safe, a compilation error would just have been more useful than a runtime crash in most cases)
There are several ways to safely provide array bounds check hints to the Rust compiler, in-fact there's a whole cookbook. But for many cases, yep, runtime check.
What's the alternative?
https://play.rust-lang.org/?version=stable&mode=debug&editio...
https://play.rust-lang.org/?version=stable&mode=debug&editio...
Catch the panic & unwind, safe program execution continues. Fundamentally impossible in Fil-C.
I do not know how Fil-C handles this, but it could raise a signal that one can then catch.
Seems like a niche use case. If it needs code to handle, it's also not apples to apples...
It's an apple to non-existent-apple comparison. Fil-C can't handle it even with extra code because Fil-C provides no recovery mechanism.
I also don't think it's that niche a use case. It's one encountered by every web server or web client (scope exception to single connection/request). Or anything involving batch processing, something like "extract the text from these 10k PDFs on disk".
Sure, it's not implemented in Fil-C because it is very new and the point of it is to improve things without extensive rewrites.
Generally, I think one could want to recover from errors. But error recovery is something that needs to be designed in. You probably don't want to catch all errors, even in a loop handling requests for an application. If your application isn't designed to handle the same kinds of memory access issues as we're talking about here, the whole thing turns into non-existent-apples to non-existent-apples lol.
.get() will bounds check and the compiler will optimize that away if it can prove safety at compile time. That leaves you 3 options made available in Rust:
- Explicitly unsafe
- Runtime crash
- Runtime crash w/ compile time avoidence when possible
For some things the just-crash is ok, like cli usage of curl
Rust also has run-time crash checks in the form of run-time array bounds checks that panic. So let us not pretend that Rust strictly catches everything at compile-time.
It’s true that, assuming all things equal, compile-time checks are better than run-time. I love Rust. But Rust is only practical for a subset of correct programs. Rust is terrible for things like games where Rust simply can not prove at compile-time that usage is correct. And inability to prove correctness does NOT imply incorrectness.
I love Rust. I use it as much as I can. But it’s not the one true solution to all things.
Not trying to be a Rust advocate and I actually don't work in it personally.
But Rust provides both checked alternatives to indexed reads/writes (compile time safe returning Option<_>), and an exception recovery mechanism for out-of-bounds unsafe read/write. Fil-C only has one choice which is "crash immediately".
What makes you think that one can not add an explicit bound check in C?
>And inability to prove correctness does NOT imply incorrectness.
And inability to prove incorrectness does NOT imply correctness. I think most Rust users don't understand either, because of the hype.
Thanks for the love man!
> "rewrite it in rust for safety" just sounds stupid
To be fair, Fil-C is quite a bit slower than Rust, and uses more memory.
On the other hand, Fil-C supports safe dynamic linking and is strictly safer than Rust.
It's a trade off, so do what you feel
Minor nitpick. Or confusion on my part. In the filc_malloc function the call to calloc doesn't seem to allocate enough memory to store an AllocationRecord for each location in visible_bytes. Should it be:
Note, I'm not the author of the OP.
I am the author of Fil-C
If you want to see my write-ups of how it works, start here: https://fil-c.org/how
Thanks, I did confuse you for the author of the article. Your InvisiCaps explanation is clearer than this "simplified" one.
> Fil-C is one of the most unrated projects I've ever seen
When's the last time you told a C/C++ programmer you could add a garbage collector to their program, and saw their eyes light up?
Exactly, the Venn diagram of programmers using c/c++ and programmers who can use a garbage collector for their workload is two circles.
Except for:
- Me. I'm a C++ programmer.
- Any C++ programmer who has added a GC to their C++ program. (Like the programmers who used the web browser you're using right now.)
- Folks who are already using Fil-C.
I’m also a C++ programmer, I can’t even use half of the C++ stdlib for real time thread work, I certainly can’t use a GC.
There are many C++ programmers and we are not the same!
My original foray into GCs was making real time ones, and the Fil-C GC is based on that work. I haven’t fully made it real time friendly (the few locks it has aren’t RT-friendly) but if I had more time I could make it give you hard guarantees.
It’s already full concurrent and on the fly, so it won’t pause you
It really doesn't, Rust is a better language.
Fil-C has two major downsides: it slows programs down and it doesn't interoperate with non-Fil-C code, not even libc. That second problem complicates using it on systems other than Linux (even BSDs and macOS) and integrating it with other safe languages.
You’re not wrong but both problems could be alleviated by sending patches :-)
I would never say it's impossible, and you've done some amazing work, but I do wonder if the second problem is feasibly surmountable. Setting aside cross-language interop, BYOlibc is not really tolerated on most systems. Linux is fairly unique here with its strongly compatible syscall ABI.
You're right that it's challenging. I don't think it's infeasible.
Here's why:
1. For the first year of Fil-C development, I was doing it on a Mac, and it worked fine. I had lots of stuff running. No GUI in that version, though.
2. You could give Fil-C an FFI to Yolo-C. It would look sort of like the FFIs that Java, Python, or Ruby do. So, it would be a bit annoying to bridge to native APIs, but not infeasible. I've chosen not to give Fil-C such an FFI (except a very limited FFI to assembly for constant time crypto) because I wanted to force myself to port the underlying libraries to Fil-C.
3. Apple could do a Fil-C build of their userland, and MS could do a Fil-C build of their userland. Not saying they will do it. But the feasibility of this is "just" a matter of certain humans making choices, not anything technical.
> it slows programs down
Interesting, how costly would be hardware acceleration support for Fil-C code.
I think there's two main avenues for hardware acceleration: pointer provenance and garbage collection. The first dovetails with things like CHERI [1] but the second doesn't seem to be getting much hardware attention lately. It has been decades since Lisp Machines were made, and I'm not aware of too many other architectures with hardware-level GC support. There are more efficient ways to use the existing hardware for GC though, as e.g. Go has experimented with recently [2].
[1]: https://en.wikipedia.org/wiki/Capability_Hardware_Enhanced_R...
[2]: https://go.dev/blog/greenteagc
It makes more sense for new software to be written in Rust, rather than a full rewrite of existing C/C++ software to Rust in the same codebase.
Fil-C just does the job with existing software in C or C++ without an expensive and bug riddled re-write and serves as a quick protection layer against the common memory corruption bugs found in those languages.
Fil-C is much slower, no free lunch, if you want the language to be fast and memory safe you need to add restrictions to allow proper static analysis of the code.
I write C++ for my job everyday, and claiming Fil-C does the same thing as Rust (and that people who do rewrites in Rust are stupid) sounds braindead.
I love Fil-C. It's underrated. Not the same niche as Rust or Ada.
This is yet another variant of the "fat pointers" technique, which has been implemented and rejected many times due to either insufficient security guarantees, inability to cross non-fat ABI boundaries, or the overhead it introduces.