Firefighter.sh gives a LLM root access to your server to constantly patch it as soon as a CVE drops, with zero confirmation or guardrails.
It is a terrible idea.
Warning: Do not run this. I am the author and I am not running this on any of my prod servers.
I am however running it on clones of my production servers (with all secrets / PII wiped), and I'm going to monitor it to see how it does in the coming months.
>I'm going to monitor it to see how it does in the coming months
Will be very interesting to see how this pans out. Would be interested to see how many servers it ran on, how many applied which fixes, how many broke something, how long each took to fix/break and on average, success rates of different models...
It does, technically, add another attack surface because "Posts, advisories, repository content and command output can contain hostile instructions..." Interesting to see if it's any better at understanding and applying "hey guy's it's fixed all you have to do is" posts. And what if the official sources are genuinely reccomending disconnecting until a patch is posted? Would it choose to 'sleep 60', check for fix, repeat? It would be directly susceptible to supply chain attacks, no? Like, if someone cracked a distros website, socials, git, etc and posted a "download critical fix", would it wait to see if say maintainers are commenting on it in other channels?
I could see this method being potentially useful serving a client running on the server it's monitoring with backups and extensive relevant tests. It could apply fix, get server response, continue or apply fix, no response, roll to last good in a way that it can't do if it's running on the machine it borked. (though, aren't there ways to do that at the kernel level where it rolls back if it fails to boot? not familiar)
> It does, technically, add another attack surface because [of prompt injection]
Heavily agree. It's really a bet of whether or not LLMs are smart enough now to navigate that on their own. I try and make that clear in the FAQ: https://firefighter.sh/faq/
I'll post results in a couple of months wrt what happens!
Firefighter.sh gives a LLM root access to your server to constantly patch it as soon as a CVE drops, with zero confirmation or guardrails.
It is a terrible idea.
Warning: Do not run this. I am the author and I am not running this on any of my prod servers.
I am however running it on clones of my production servers (with all secrets / PII wiped), and I'm going to monitor it to see how it does in the coming months.
>I'm going to monitor it to see how it does in the coming months
Will be very interesting to see how this pans out. Would be interested to see how many servers it ran on, how many applied which fixes, how many broke something, how long each took to fix/break and on average, success rates of different models...
It does, technically, add another attack surface because "Posts, advisories, repository content and command output can contain hostile instructions..." Interesting to see if it's any better at understanding and applying "hey guy's it's fixed all you have to do is" posts. And what if the official sources are genuinely reccomending disconnecting until a patch is posted? Would it choose to 'sleep 60', check for fix, repeat? It would be directly susceptible to supply chain attacks, no? Like, if someone cracked a distros website, socials, git, etc and posted a "download critical fix", would it wait to see if say maintainers are commenting on it in other channels?
I could see this method being potentially useful serving a client running on the server it's monitoring with backups and extensive relevant tests. It could apply fix, get server response, continue or apply fix, no response, roll to last good in a way that it can't do if it's running on the machine it borked. (though, aren't there ways to do that at the kernel level where it rolls back if it fails to boot? not familiar)
> It does, technically, add another attack surface because [of prompt injection]
Heavily agree. It's really a bet of whether or not LLMs are smart enough now to navigate that on their own. I try and make that clear in the FAQ: https://firefighter.sh/faq/
I'll post results in a couple of months wrt what happens!