The advice here is really great (as I'd expect from Hugo), but as the maintainer of zizmor I want to note some caveats with the current iteration of workflow lockfiles:
- They only lock normal action calls. If your workflows invoke other reusable workflows, you need to keep those hash-pinned manually for now.
- Composite actions are not themselves locked. Instead, they get locked transitively through their callsites i.e. the workflows that call them. This makes any individual caller that uses lockfiles safe, but it also makes the current `gh actions-lock` default unsafe (since it removes hash-pins from the composite action itself, meaning that any external caller that invokes the action loses their transitive pinning).
- Other forms of `uses:` clauses, like Docker images, are also not currently locked via the lockfile. And the same for forms of interior mutability (like an action running `npx ...` under the hood, although the latter is fully outside of GitHub's control to prevent. But these mean that you can't assume that a fully locked Actions setup is in fact 100% immutable.
I hope they address these gaps soon, because it's otherwise an excellent (and long-awaited) feature.
And, the one often forgotten: `runs-on:` also refers to an image, but in practice a huge amount of people will just set it to `ubuntu-latest` and accept whatever GH happens to include that day. In fact, I don't think it is even possible to refer to a specific runner image revision by digest...
Quoting someone else's blog I've forgotten but you're trading the trust in the repo owner not moving a tag for trusting whoever merges the commit hash to have checked that's a release in the repo you want and not _some_ forks commit. All forks on github are the same repo you can pin any commit, merged upstream or not.
> As the consumer of an action that becomes really hard to find out unless I take the whole codebase apart.
Maybe it's just me, but I only use (public) Actions I'm able to review in an hour or less, meaning less than 5K LOC or so. Not sure how people are able to just "consume" 3rd party GitHub Actions and not even review what exactly they do, how are people comfortable with the risks this opens up? It's literally supply-chain wild west, but seemingly people continue like it's 2010.
These days setting up a proper system on your build orchestrator of choice is pretty easy, cheaper, and you basically only lose a tiny bit of UI niceness if they report to github anyway
Just about anything else, tbh. Jenkins is the option I would go for, but that’s a reflection of my personal preferences for self-hosting. Self-hosting has gotten way easier since LLMs can debug things so well, so I’m self-hosting more than ever before even though I have less time available.
You don’t have to go Jenkins, it’s just that GitHub Actions are uniquely bad and I would avoid them.
If you are relying on the free version of GitHub actions for small projects, I would consider just running something on a NUC or Pi at home.
GHA over Jenkins is insane. Every time I’ve touched GHA, I’ve just been shocked with how bad the ecosystem is.
It feels like, I don’t know. You go to somebody’s house and they have one plate, one cup, one spork. They’re living in a 30th floor walk-up, but one of the windows is missing so there’s a giant hole in the wall.
That’s kind of the subjective experience I get when I see people using GitHub actions.
I'm not saying GHA is much better, but Jenkins is literally swiss cheese, it has vulnerabilities every day of the week. Even just as an internal service not exposed to the internet it's dangerous.
A Pi is not going to make your Windows and macOS builds. It's also going to have a miserable time running the Rust compiler on your five hundred dependencies.
Here’s my take: if you don’t personally have a Mac, you should not be making Mac builds of your software.
Same for Windows.
I may be a little old-school about this. That’s fine. I think most people out there who are trying to support multiple platforms with their code should be going one of three directions—(1) some kind of web app, (2) an open-source tool where end-users compile it or run it in their Python interpreter or Node.js or whatever, or (3) a pre-built binary for systems they personally run at home.
I also am not writing out, like, an essay on how to replace GitHub actions for every possible use case that people have GitHub actions for. I am just providing some suggestions for one possible direction. If you get off GitHub actions, you are going to end up with something that is missing some of the features that GitHub actions, maybe some features that you like. The same is true in the reverse—if you switch to GitHub actions, you will obviously lose features that were present in whatever other system you had before. GitHub actions is, after all, completely terrible and switching to it would just be a kind of disaster in a bunch of different ways.
I don’t like to put this kind of wall-of-text disclaimer in comments on HN, and hopefully y’all can figure this stuff out without the wall-of-text disclaimer. It would be kind of… absurd… perhaps, to expect some random commentator on Hacker News to give you exactly the answer that satisfies your needs, unless you put those needs in a question.
Instead I limited myself to two suggestions. If you don’t like them, then figure something else out yourself. You will not have trouble making something better than GitHub actions.
> if you don’t personally have a Mac, you should not be making Mac builds of your software
Apple seem to believe this extremely strongly and keep finding new ways to prevent you doing it.
I basically agree, and at work we have a Jenkins master + cluster of build machines. But this does incur substantial sysadmin overhead even if we're using Ansible and all the Windows ones are VMs. (Mac, again, hates being either virtualized or stuck in a rack, and will occasionally insist on someone using the physical display and keyboard on the actual machine to perform updates)
Are people finding a real need for importing and updating git actions? Aren't they usually just running a one line bash command? I'd just copy paste the action in to your config so it's fully locked and under your control rather than importing some opaque script from a no name github user.
A lot of things that should be one bash command turn out to be dozens of lines of incantations and handling edge cases. Microsoft I'm looking in your direction.
I undrrstand this looks cleaner and more intuitive, but you still have the risk of running a workflow with malicious/compromised actions, like manually, or locally with act, if you do not verify the lockfile as part of the workflow or preceded by an extra workflow step. This should be something that the runner ideally should handle (as part of the platform). This still is a workaround, just like the sha pinning. I only use vendor upstream actions, like github's or forks in our own org.
Presumably it’s for the transitive dependencies. You pin action X, but that (third party) action doesn’t correctly pin the SHA of action Y, so you still get owned if the mutable tag gets moved to a compromised commit.
But with a lockfile, direct and indirect dependencies both get pinned. (I presume, and haven’t checked yet, but otherwise this feature would be useless)
It does not. Obviously if you lock a “foo/setup-latest-go” action, it’s still going to (and you’d probably expect it to) pull in the latest go. Actions are expected to start running though. Like GitHub isn’t gonna do an `npm install` for you, it’ll just run the entry point of the action, and the entry point can do whatever it was written to do.
I remember reading (but haven’t verified) that even using a specific SHA instead of a tag isn’t a guarantee with GitHub Actions, because the Action itself may have a dependency which does not pin to a SHA, thus making you vulnerable all the same. This lock file strategy does not appear to guard agains that, it seems to be what we had before but trades a bit more readability in one file for an extra (more complex) lock file. Not a trade-off I want to make (but you do you).
Personally I found the risk of running someone else’s GitHub Action in my repo to not be worth it. Even GitHub’s own Action to checkout repos is way more complex than I need it to be and last I checked had dependencies which could be targeted. I opt instead to not run any third-party Actions (not even GitHub’s official ones) in my repos.
I did read the blog post. More than once, in search for info on transitive dependencies. It doesn’t provide it. You’re right that I should’ve clicked through the official blog post, I got link blindness. I did find it strange that transitive dependencies weren’t included, which is why I repeatedly said it seemed it didn’t. Thank you for the correction.
Your snark is unwarranted, though, since the crux of the comment is not using any Action you haven’t built. Lockfiles don’t solve the complexity and trust problem.
Sorry, the snark was meant to be more general. Your comment was one of ~5 of people that apparently didn't pick up on the transitive locking, and repeatin now-outdated information. I do think the author did a bad job of highlighting it, so I see how it can be missed.
> Lockfiles don’t solve the complexity and trust problem.
They don't solve it on their own, but they are an important building block, and allow GHA to take the same shape that other package ecosystems have (e.g. npm, Cargo), where trust issues can be worked around to some degree with a combination of consumption-delay + automated security review. Of course, going with a smaller surface area is always preferrable.
Fair. I know what it’s like to be frustrated with a general trend, and unfortunately we have to pick one comment to reply to.
> They don't solve it on their own, but they are an important building block
Agreed, it is a good step and I understand why people would want to do it. But I also stand by saying “not a trade-off I want to make (but you do you)”. I mean it in every sense: I still don’t want to do it for myself, but in no way am I claiming my preference is right for everyone.
If that is really the "entire point" of the article, it's strange that the article doesn't mention it at all. It simply says you should do a thing differently than the way you used to do it, without explanation. It links to the official GitHub blog post, which, it turns out, does mention transitivity -- but again, TFA itself does not even hint that this "entire point" will be addressed in that linked article.
As I see it, TFA's only value is that link to a better article.
Yes, I scratched my head trying to understand what this accomplishes. It just puts a tag back instead of a SHA but now moves the SHA (locked in) to a separate file that now needs to be verified ?
I fail to see how this is better or needed.
SHA pinning on GitHub does not always imply account pinning, even where the account name is explicitly specified along with it.
So some of the extra steps are meaningful. Though I am still not convinced they could not have been implemented in a deliberately backwards-incompatible way to Github action in general, keeping the syntax and removing only the special case that is never both intentional and non-malicious at the same time.
at this point im seriously pondering how to build a dvcs code forge thats not dog shit, versions its own state, and actually is semantically grounded rather than a rails app gone awry
Shouldn’t we all just be vibe coding our own actions instead of using the marketplace ones? Is SaaS dead yet, or what?
/s
This is a step in the right direction, but it’s not even close to a complete solution. And what am I doing installing a plugin to use this? It should just be part of GH, so I can install that and not have yet another source of dependency management to lock down.
So instead of specifying a SHA1 hash of some commit and change it rarely enough to just check manually, now every developer needs to unsafe-eval some 476 transitive dependencies of the helper that fixes the lockfile? I wonder at which point it gets easier to just have a nice stdlib again, and not deal with tons and tons of version changes in packages across 3 different language ecosystems.
The advice here is really great (as I'd expect from Hugo), but as the maintainer of zizmor I want to note some caveats with the current iteration of workflow lockfiles:
- They only lock normal action calls. If your workflows invoke other reusable workflows, you need to keep those hash-pinned manually for now.
- Composite actions are not themselves locked. Instead, they get locked transitively through their callsites i.e. the workflows that call them. This makes any individual caller that uses lockfiles safe, but it also makes the current `gh actions-lock` default unsafe (since it removes hash-pins from the composite action itself, meaning that any external caller that invokes the action loses their transitive pinning).
- Other forms of `uses:` clauses, like Docker images, are also not currently locked via the lockfile. And the same for forms of interior mutability (like an action running `npx ...` under the hood, although the latter is fully outside of GitHub's control to prevent. But these mean that you can't assume that a fully locked Actions setup is in fact 100% immutable.
I hope they address these gaps soon, because it's otherwise an excellent (and long-awaited) feature.
And, the one often forgotten: `runs-on:` also refers to an image, but in practice a huge amount of people will just set it to `ubuntu-latest` and accept whatever GH happens to include that day. In fact, I don't think it is even possible to refer to a specific runner image revision by digest...
Quoting someone else's blog I've forgotten but you're trading the trust in the repo owner not moving a tag for trusting whoever merges the commit hash to have checked that's a release in the repo you want and not _some_ forks commit. All forks on github are the same repo you can pin any commit, merged upstream or not.
IMO Github should make Immutable tags mandatory to publish on https://github.com/marketplace
Actions lockfiles prevent this. They won't allow you to lock an impostor commit, which is the right default.
(The previous way to provide general actions immutability, i.e. hash-pinning, had this issue and thus required additional checks.)
One could write an action that executes a shell script that does docker pull abc:latest.
As the consumer of an action that becomes really hard to find out unless I take the whole codebase apart.
> As the consumer of an action that becomes really hard to find out unless I take the whole codebase apart.
Maybe it's just me, but I only use (public) Actions I'm able to review in an hour or less, meaning less than 5K LOC or so. Not sure how people are able to just "consume" 3rd party GitHub Actions and not even review what exactly they do, how are people comfortable with the risks this opens up? It's literally supply-chain wild west, but seemingly people continue like it's 2010.
Why not make the marketplace published tags immutable instead like others do?
That’s exactly what lockfiles are intended to avoid!
I almost forgot about that completely outrageous treatment of forks -- just bonkers.
I think my main advice is to avoid GH actions at all, unless avoiding them is somehow painful. There is just a long bad history here.
I just replaced all of my github CI/CD with woodpecker running on a desktop on-premise.
Works completely fine and no github bill.
These days setting up a proper system on your build orchestrator of choice is pretty easy, cheaper, and you basically only lose a tiny bit of UI niceness if they report to github anyway
Only if your not on GH free, which I use for my OSS projects for example.
What would you suggest instead?
cron and bash
Just about anything else, tbh. Jenkins is the option I would go for, but that’s a reflection of my personal preferences for self-hosting. Self-hosting has gotten way easier since LLMs can debug things so well, so I’m self-hosting more than ever before even though I have less time available.
You don’t have to go Jenkins, it’s just that GitHub Actions are uniquely bad and I would avoid them.
If you are relying on the free version of GitHub actions for small projects, I would consider just running something on a NUC or Pi at home.
Jenkins over GHA is insane.
GHA over Jenkins is insane. Every time I’ve touched GHA, I’ve just been shocked with how bad the ecosystem is.
It feels like, I don’t know. You go to somebody’s house and they have one plate, one cup, one spork. They’re living in a 30th floor walk-up, but one of the windows is missing so there’s a giant hole in the wall.
That’s kind of the subjective experience I get when I see people using GitHub actions.
I'm not saying GHA is much better, but Jenkins is literally swiss cheese, it has vulnerabilities every day of the week. Even just as an internal service not exposed to the internet it's dangerous.
A Pi is not going to make your Windows and macOS builds. It's also going to have a miserable time running the Rust compiler on your five hundred dependencies.
Here’s my take: if you don’t personally have a Mac, you should not be making Mac builds of your software.
Same for Windows.
I may be a little old-school about this. That’s fine. I think most people out there who are trying to support multiple platforms with their code should be going one of three directions—(1) some kind of web app, (2) an open-source tool where end-users compile it or run it in their Python interpreter or Node.js or whatever, or (3) a pre-built binary for systems they personally run at home.
I also am not writing out, like, an essay on how to replace GitHub actions for every possible use case that people have GitHub actions for. I am just providing some suggestions for one possible direction. If you get off GitHub actions, you are going to end up with something that is missing some of the features that GitHub actions, maybe some features that you like. The same is true in the reverse—if you switch to GitHub actions, you will obviously lose features that were present in whatever other system you had before. GitHub actions is, after all, completely terrible and switching to it would just be a kind of disaster in a bunch of different ways.
I don’t like to put this kind of wall-of-text disclaimer in comments on HN, and hopefully y’all can figure this stuff out without the wall-of-text disclaimer. It would be kind of… absurd… perhaps, to expect some random commentator on Hacker News to give you exactly the answer that satisfies your needs, unless you put those needs in a question.
Instead I limited myself to two suggestions. If you don’t like them, then figure something else out yourself. You will not have trouble making something better than GitHub actions.
> if you don’t personally have a Mac, you should not be making Mac builds of your software
Apple seem to believe this extremely strongly and keep finding new ways to prevent you doing it.
I basically agree, and at work we have a Jenkins master + cluster of build machines. But this does incur substantial sysadmin overhead even if we're using Ansible and all the Windows ones are VMs. (Mac, again, hates being either virtualized or stuck in a rack, and will occasionally insist on someone using the physical display and keyboard on the actual machine to perform updates)
GitHub actions is bad, but if you keep things mostly in locally verifiable scripts it's really not that bad.
And yes, you should have a Mac for local testing of you target OSX, but it's still useful to have an easy way to run a matrix of platforms in CI.
Jenkins is fine but hard to call it better than GH actions, subjectively.
A Pi can run MinGW or Clang with a Win32 target. Even running MSVC under Wine is technically possible.
The whole system is just bad. Even Jenkins is better. Or a shell script in a one minute cronjob.
Are people finding a real need for importing and updating git actions? Aren't they usually just running a one line bash command? I'd just copy paste the action in to your config so it's fully locked and under your control rather than importing some opaque script from a no name github user.
Github actions are not that simple. Not even close
Still, I prefer to copy their action.yml into my repo.
(I simply put a comment with the upstream URL in my repo in case I want to update in the future.)
A lot of things that should be one bash command turn out to be dozens of lines of incantations and handling edge cases. Microsoft I'm looking in your direction.
For those things an action helps.
I undrrstand this looks cleaner and more intuitive, but you still have the risk of running a workflow with malicious/compromised actions, like manually, or locally with act, if you do not verify the lockfile as part of the workflow or preceded by an extra workflow step. This should be something that the runner ideally should handle (as part of the platform). This still is a workaround, just like the sha pinning. I only use vendor upstream actions, like github's or forks in our own org.
say more about how you would envision the runner handling this? An allowlist of actions?
The runner should execute gh actions-lock --verify automatically.
Are there clear reasons to prefer this over just pinning a commit SHA?
Presumably it’s for the transitive dependencies. You pin action X, but that (third party) action doesn’t correctly pin the SHA of action Y, so you still get owned if the mutable tag gets moved to a compromised commit.
But with a lockfile, direct and indirect dependencies both get pinned. (I presume, and haven’t checked yet, but otherwise this feature would be useless)
You can keep the information provided by the version number, and if the maintainer is using real SemVer, it's easier to upgrade.
> You can keep the information provided by the version number
The old way of just writing this in a comment on the same line seems fine to me -- intuitive and much more concise.
> if the maintainer is using real SemVer, it's easier to upgrade
Could you elaborate on this? I didn't see how yet.
No, I'm still keep using 'pinact run -u', and some dependabot actions to remind me on updates.
Does this prevent one of the locked dependencies not locking their own dependencies and pulling in a malicious and replaced tag?
It does not. Obviously if you lock a “foo/setup-latest-go” action, it’s still going to (and you’d probably expect it to) pull in the latest go. Actions are expected to start running though. Like GitHub isn’t gonna do an `npm install` for you, it’ll just run the entry point of the action, and the entry point can do whatever it was written to do.
I think they mean specifically sub-actions as dependencies.
big fan of actions-up for doing this: https://github.com/azat-io/actions-up
I remember reading (but haven’t verified) that even using a specific SHA instead of a tag isn’t a guarantee with GitHub Actions, because the Action itself may have a dependency which does not pin to a SHA, thus making you vulnerable all the same. This lock file strategy does not appear to guard agains that, it seems to be what we had before but trades a bit more readability in one file for an extra (more complex) lock file. Not a trade-off I want to make (but you do you).
Personally I found the risk of running someone else’s GitHub Action in my repo to not be worth it. Even GitHub’s own Action to checkout repos is way more complex than I need it to be and last I checked had dependencies which could be targeted. I opt instead to not run any third-party Actions (not even GitHub’s official ones) in my repos.
Does nobody read the blog post and liked Github release annoucment anymore?
This is a new first-party feature by Github to allow locking of transitive dependencies.
I did read the blog post. More than once, in search for info on transitive dependencies. It doesn’t provide it. You’re right that I should’ve clicked through the official blog post, I got link blindness. I did find it strange that transitive dependencies weren’t included, which is why I repeatedly said it seemed it didn’t. Thank you for the correction.
Your snark is unwarranted, though, since the crux of the comment is not using any Action you haven’t built. Lockfiles don’t solve the complexity and trust problem.
Sorry, the snark was meant to be more general. Your comment was one of ~5 of people that apparently didn't pick up on the transitive locking, and repeatin now-outdated information. I do think the author did a bad job of highlighting it, so I see how it can be missed.
> Lockfiles don’t solve the complexity and trust problem.
They don't solve it on their own, but they are an important building block, and allow GHA to take the same shape that other package ecosystems have (e.g. npm, Cargo), where trust issues can be worked around to some degree with a combination of consumption-delay + automated security review. Of course, going with a smaller surface area is always preferrable.
> Sorry, the snark was meant to be more general.
Fair. I know what it’s like to be frustrated with a general trend, and unfortunately we have to pick one comment to reply to.
> They don't solve it on their own, but they are an important building block
Agreed, it is a good step and I understand why people would want to do it. But I also stand by saying “not a trade-off I want to make (but you do you)”. I mean it in every sense: I still don’t want to do it for myself, but in no way am I claiming my preference is right for everyone.
Cosign. It’s the entire point of this article and would be obvious if someone read
If that is really the "entire point" of the article, it's strange that the article doesn't mention it at all. It simply says you should do a thing differently than the way you used to do it, without explanation. It links to the official GitHub blog post, which, it turns out, does mention transitivity -- but again, TFA itself does not even hint that this "entire point" will be addressed in that linked article.
As I see it, TFA's only value is that link to a better article.
So "SHA pinning with version reference in a comment" with extra steps, got it
Yes, I scratched my head trying to understand what this accomplishes. It just puts a tag back instead of a SHA but now moves the SHA (locked in) to a separate file that now needs to be verified ? I fail to see how this is better or needed.
You can still use the SHA if you want to. What it achieves is transitive pinning (which this post doesn't properly highlight).
Lack of transitive pinning support was the biggest supply-chain security criticism against GHA and I'm glad they are finally tackling it!
SHA pinning on GitHub does not always imply account pinning, even where the account name is explicitly specified along with it.
So some of the extra steps are meaningful. Though I am still not convinced they could not have been implemented in a deliberately backwards-incompatible way to Github action in general, keeping the syntax and removing only the special case that is never both intentional and non-malicious at the same time.
Enforced through GitHub Actions that is :)
at this point im seriously pondering how to build a dvcs code forge thats not dog shit, versions its own state, and actually is semantically grounded rather than a rails app gone awry
If GitHub really cared about security they could fix this tomorrow by declaring any tag linked to an action immutable.
They don't because they don't.
Anybody who's ever moved a git tag knows the ergonomics of doing so are so awful that you're obviously not supposed to be doing it.
What tools did you use to get that awfulness?
Shouldn’t we all just be vibe coding our own actions instead of using the marketplace ones? Is SaaS dead yet, or what?
/s
This is a step in the right direction, but it’s not even close to a complete solution. And what am I doing installing a plugin to use this? It should just be part of GH, so I can install that and not have yet another source of dependency management to lock down.
So instead of specifying a SHA1 hash of some commit and change it rarely enough to just check manually, now every developer needs to unsafe-eval some 476 transitive dependencies of the helper that fixes the lockfile? I wonder at which point it gets easier to just have a nice stdlib again, and not deal with tons and tons of version changes in packages across 3 different language ecosystems.
ctrl + f "zizmor" 8 found
I endose this article.