> I almost always leave a comment on each PR I review, even just observations: “This class is getting big, we might want to consider adding a presenter,” or praise: “Thanks for cleaning this up!”
Things like that I'd much rather leave as comments in the code, rather than dangling as off-hand things in some unrelated PR. Probably no one will see those PR comments unless they go splunking for some reason, but having them right in the code is much more evident to all the people passing through, or even reminding yourself in the future too.
In general I feel like PR reviews are basically reviews happening too late, and if there is a lot of stuff to review and agree upon in the PR, you'll be better off trying to reduce that up front by discussing and noting design decisions before the work even starts. If there is so many unknowns first, do one step to figure out how it can be done, then agree on that, again before starting the actual work.
That leads to PRs basically just being spot-checking, not deep analysis nor even space for nitpicks. If you as a reviewer spots things after the implementation rather in the discussion beforehand, that ends up being on both of you, instead of just the implementer who tried to move along to finish the thing.
You might be approaching PR comments differently than I've seen. When a comment is something to be addressed, it's either put into a new development task (i.e. on something like Jira), or it is completed before the PR merges. I'm not sure that having comments in the code surfaces that information in a useful manner. The code is for the code, not for what the code could be. The comments on what it could be should be handled outside the code at a different abstraction layer.
Agreed. Putting comments in code is good for adding context around the code, but the actual action item needs to be tracked in the same place that all other action items and issues are tracked.
An exception would be for information that doesn't yet qualify as an action item, but could become an action item if someone changes something in the code. Like if removing a conditional check would trigger the need for some other work or a refactor. Then it's good to put it close to the code so anyone touching that code will know they need to make the action items if they go down that route.
Perhaps in a small codebase without issue tracking this might be something leverage. It's just not reasonable with 100k+ lines of enterprise grade code.
> In general I feel like PR reviews are basically reviews happening too late, and if there is a lot of stuff to review and agree upon in the PR, you'll be better off trying to reduce that up front by discussing and noting design decisions before the work even starts.
I agree in general and tried to push for a more iterate approach in our team.
However, my fear is that this would multiply the effort because it's very easy to discuss many things that do not matter or turn out later to not matter.
It seems it's easier for people to only discuss the important stuff when presented as an actual implementation.
We are talking tendencies here, of course, general design decision must always be done beforehand.
> It seems it's easier for people to only discuss the important stuff when presented as an actual implementation.
LLMs help a lot here, create two prototypes with both designs, compare them together :) Could even evaluate how ergonomic some future potential change is, to see how it holds up in practice.
I used to use pen and paper to essentially do the same, minus having real code and instead just talking concepts, but it does suffer from the issue that some need to be confronted with code in front of them to visualize things better for themselves.
I'm not following this. PRs are the first time your reviewers have seen that change, so there is no opportunity downstream to do the things you're suggesting.
You're essentially suggesting pre-PRs, but it is circular, since those same pre-PRs would have the same criticism.
PRs are about isolated core changes, not feature or system design. They answer how not why.
Usually by the time a PR has been submitted it's too late to dig into aspects of the change that come from a poor understanding of the task at hand without throwing out the PR and creating rework.
So it's helpful to shift left on that and discuss how you intend to approach the solution. Especially for people who are new to the codebase or unfamiliar with the language and, thanks to AI, show little interest in learning.
Obviously not for every situation, but time can be saved by talking something through before YOLOing a bad PR.
Yes, it should be cheap to throw out any individual PR and rewrite it from scratch. Your first draft of a problem is almost never the one you want to submit anyway. The actual writing of the code should never be the most complicated step in any individual PR. It should always be the time spent thinking about the problem and the solution space. Sometimes you can do a lot of that work before the ticket, if you're very familiar with the codebase and the problem space, but for most novel problems, you're going to need to have your hands on the problem itself to get your most productive understanding of them.
I'm not saying it's not important to discuss how you intend to approach the solution ahead of time, but I am saying a lot about any non-trivial problem you're solving can only be discovered by attempting to solve it. Put another way: the best code I write is always my second draft at any given ticket.
More micromanaging of your team's tickets and plans is not going to save you from team members who "show little interest in learning". The fact that your team is "YOLOing a bad PR" is the fundamental culture issue, and that's not one you can solve by adding more process.
I don't disagree that a practical spike is a good way to grasp a novel problem (or work with a lack of internal knowledge because it's legacy code) but there is still something to be said for attempting to work things out in the abstract too, and not necessarily by adding process, but by redeveloping that internal knowledge and getting familiar with the business domain.
In a greenfield project I will have a lot of patience for a team that doesn't grasp the problem space too well yet, and needs to feel around it by experimenting and prototyping. You have to encourage that or you might not even be building anything innovative.
For the longer term legacy project then the team can't really afford to have people going down rabbit holes and it's more beneficial to approach things in the abstract and reduce the problem as much as possible. Especially with junior or mid-level engineers who can see an old codebase as a goldmine for refactoring if left unattended.
As for the fundamental culture issue... maybe. AI increases the frequency of low quality PRs and puts a bigger burden on the reviewer. I can live with this in the short term if people take lessons from it and keep building up their own skillset. I feel this issue is not unique to my team and LLM-driven development is still novel enough that we're all figuring out the best way to tackle it.
Asking a more junior developer or someone who "show little interest in learning" to discuss their approach with you before they've spent too much time on the problem, especially if you expect them to take the wrong approach seems like the right way to do things.
Throwing out a PR of someone who doesn't expect it would be quite unpleasant, especially coming from someone more senior.
This is how I try to approach it. I don't think it's a new thing for a new hire to come in hot and try to figure things out themselves rather than spending time with the team. Or getting lost down rabbit holes.
> PRs are the first time your reviewers have seen that change, so there is no opportunity downstream to do the things you're suggesting.
Yes, but I'm arguing for that it shouldn't be the first time they hear about that this change is planned and happening, and their input should have taken into account before the PR is even opened, either upfront/before or early in development. This eliminates so many of the typical PR reviews/comments.
Figure out where you are going before you start going there, instead of trying to course correct after people already walked places.
I think it's a fine line to walk. At my job what we do is discuss any complex implementation or risky change or blockers in the dev eng meeting. For smaller stuff, or more straightforward solutions, we don't bring it up. If you make it a hard rule to first discuss all tickets, it just seems draconian.
Code review is specifically for code quality, more lower level stuff.
When I review a PR, I don't add comments to the code. If I think something needs to be commented: I comment on the PR and reject the PR.
If I think --as is discussed in the article-- that the comment "would be nice, but is not strictly necessary", then I comment on this on the PR and approve the PR.
> If you as a reviewer spots things after the implementation rather in the discussion beforehand, that ends up being on both of you, instead of just the implementer who tried to move along to finish the thing
This is accurate, but it's still an important check in the communication loop. It's not all that uncommon for two engineers to discuss a problem, and leave the discussion with completely different mental models of the solution.
This one resonates, because it's largely how I comment on PRs too!
One thing not mentioned is how important it is to acknowledge the comments too. People are taking their time to review your PR, and not even giving a reaction will make the commenter question whether or not it was even received. I'm not looking to throw my thoughts out into the aether. That's what microblogging platforms are for!
I can't tell you how many times I got no response/acknowledgement on a comment that actually surfaced something critical. I haven't been keeping track, but I think my comments could have prevented dozens of outages at this point. It's quite exhausting. In my own experience, the worst offenders of this are senior devs.
> Why approve, if I’ve left comments that I think are worth implementing?
>
> Because I trust my team. I know that my comments will be considered, and if they’re useful, implemented.
I do this a lot too. It's critical that PR authors don't burn that trust either. If they make substantial changes that warrant another review, I hope they do request it. Too many times in my career have colleagues just went ahead and made bad changes after my approval that I would have easily caught, merge, and things go
High trust, high alignment environments move so fast, and you know when you're in one and know when you have your colleagues' trust. It feels really good!
Isn’t a change to the code that incorporates your suggestions enough acknowledgment? Presumably the code change would be required to get your approval?
Of course! I didn't state it explicitly, but what I was referring to were comments that went completely unacknowledged in any way, be it explicit or implicit. No changes. No reaction. No "resolve comment". Radio silence.
As mentioned, I've experienced it too many times where not addressing a question/concern I put on the pull request led to outages that could have been avoided. I think it's typically a certain personality. It's not a common occurrence, but I have experienced it.
While one of the best things about PRs can be raising the floor on quality, I think the another powerful factor is forcing an opportunity to talk about the code. This article even pre-empts this confusion by acknowledging that commenting on a PR while approving it is "weird". This implies that many people don't understand the indirect benefits of the PR and think the process is all about approval and raising the quality floor.
I think raising the floor, gives diminishing returns once someone is used to the team and the code base, but the conversation always remains relevant. Sometimes teams that resist things like PRs (e.g. "they just slow us down") are actually teams that are having those conversions elsewhere (in-person, on slack, during standup or sprint planning, etc).
Totally agreed, but it always comes back to managing the average over the best. It is a lot more effective to enforce a culture of approvals than a culture of collaboration. Managers should be striving for the later, but not every team is built the same way and many in leadership just need assurance that the bare minimum is getting done.
> Additionally, some repos can be configured to automatically merge PRs when all requirements are met, one of which might be your approval.
If anyone at GitHub is reading this, I’d love a fourth checkbox in the “leave a review” modal that is “Approve but disable auto merge” (alongside Comment/Approve/Request changes)! Even just surfacing “this PR has auto merge enabled” near the Approve button would be great.
This feels like a suboptimal solution to me because I personally like to keep comments in "unresolved" state so that they remain visible and other folks can weigh in on them if they want, but in a way that doesn't block the PR. Basically I wish that GitHub would either separate the "collapsed" and "resolved" concepts, or add this "approve without merging" button.
I do the same as the author, and for my comments I make it clear what is non-blocking/blocking with Conventional Comments: https://conventionalcomments.org
To make it easier for myself to leave the CC tags ("nit(non-blocking): ", etc), I use the macOS text expander in System Settings and created mnemonics to easily insert them.
Example: If I type "+pnn", that maps to:
+p = pull request comment
n = nit
n = non-blocking
Example 2: If I type "+pc", that maps to:
+p = pull request comment
c = chore
(and it's blocking because I didn't type "+pcn", the non-blocking version).
I'm a big believer in this workflow and generally agree with all the guidelines about when to do and not do this.
Code review has value, but I don't think we are always honest about the costs. At most places I've worked, there has been an informal understanding that code should be reviewed within 24 hours or so, but there is a world of difference between getting a review within 2 hours and 23 hours.
If you have to go back and forth a second time, it's so much more likely that the approval comes much later due to straddling end-of-day in someone's timezone.
Tangentially, if you are designing policies for code review at your org, try to think both about minimum requirements (e.g. PRs should get a first look within 24 hours) and typical requirements (e.g. most PRs should get reviewed within 2-3 hours). What typically happens is what determines overall velocity for your org, so it's much more important than whatever strict SLA policy you have. These are also fixable problems if you have targeted conversations with people. E.g. "I noticed X, Y, Z times, you had unreviewed PRs at the end of the day. Can you set aside some time to review code before EOD? Or try installing PR reminder?"
But PR discussions about lintable style issues always surprise me. The ideal solution is to add a rule in the linter. But when the team won't agree on the rule, and is open to multiple styles, the author should decide, simple! Had a team mate recently who'd block PRs over T[] vs Array<T>, forcing people to stick with Array<T> for simple types like number[] even though TypeScript's own docs and tooling push T[].
Linters often don’t provide the constraints you desire. While they’re a great (arguably essential) tool, they often are not sufficient.
When a project has well-established patterns, part of the job is to just follow the pattern, whether you like it or not, until you find a reason to 1) change the pattern everywhere, or; 2) make an exception to the pattern with intention.
It's quite simple to write a rule, e.g. for the example I shared https://typescript-eslint.io/rules/array-type. But I've never seen any reputable project arguing against T[] for simple types. In my opinion, that says a lot about the reviewer, not the contributor. If a linter rule doesn't exist, you should argue against the idiomatic convention.
Yeah, I think all style comments should be handled by either a linter/formatter or be written in a style guide. Everything else is up to personal preference.
Then, when a style comment comes up in a PR, the answer is "Oh, do you think we should add that to our style guide? If so, let's discuss that in slack. Until then, though, that's not blocking."
This is exactly the pedantry I try to avoid in my teams.
The point of a review shouldn’t be to make sure it’s exactly how the reviewer would do it. Is it safe? Does the author understand the area? Is there evidence the code has been exercised? Does it follow the major conventions?
Beyond that, I try very hard to build a culture of approvals vs nitpicking, and let the linters enforce the rest.
This is me, I think it comes from a personality of wanting to make sure I say something but generally trusting that the author has more context than I do. I float questions and nits but generally will approve if I don't see anything glaring. If I spot anything where I think "we should definitely go another direction here" I ask for changes instead of approving and make them super clear.
> In my team, my rule is: if it’s better than what’s on master, you approve and merge.
This causes unnecessary code changes later on, code changes mean new code, new code has bugs. The team should try to get it close to perfect on the first try instead. They won't, but that should be everyone's target. If that sounds impossible, then the PR was to big.
Totally agree, and I go even farther: I consider that the PR author is totally responsible of merging their PR or not, and so no comment is blocking. I can make a comment I consider blocking, but there might be very good reasons that I can't see to still merge the PR. Fixes can come in a follow-up PR. As long as everyone takes responsibility for their work, it is a well-working system.
I woke up one day and the industry is gating all code changes behind PRs w/ approvals. Anyone still able to commit to main without approval from someone else? I’m even seeing companies require multiple approvals on every PR. It’s insanity.
I’m skeptical. Software is as buggy as it ever was. I come
across teams shipping terrible quality software, where every line change was approved and reviewed. I come across teams that require every change have an approval, but don’t require 100% test coverage. I’m seeing senior engineers have to get approval from juniors for a copy change.
It’s theatre. It’s bad management. It’s a cargo cult. It isn’t actually driving code quality.
Code review is one thing. Code review is good. But requiring every change have an approval is something else.
Where I work I can and am still allowed to push to mainline directly, as are all other developers on the team.
Some new colleagues started with PRs and since then it has been a slow move towards using PRs more and more (but still not mandatory).
As of now some colleagues don't typically do PRs and push directly (minority), some decide on a case-by-case basis (I am among those, among a small majority in our team) and some have been using PRs for each and every change from day one (also a minority).
The criticisms by the opponents of PRs are as follows:
1. People relying on PRs too much causes them to propose code which is not production quality as of the PR whereas before (i.e. "no PRs") if your code was not correct, you or someone else either noticed shortly after (checking the tests on CI etc.) or it would make it into the release. I don't believe this to be generally true but sometimes thought similar when reading some PRs. The frequency of large bugs being found by code review in the PR has gone down in recent times which leads me to believe that the colleagues have adjusted their development style to come up with good quality solutions in the first attempt already in most cases.
2. Code review is hard to do in many cases. In our team this is typically resolved by going over a PR together for the “hard cases” (e.g. in a videoconference).
I think PRs are mostly worth the effort because any bug that can be avoided before the release saves a lot of downstream effort (e.g. support) and the effect of building a shared knowledge about the code page is valuable (although hard to quantify).
1. Does it work? Then ship it. This is great for early on, high-velocity where the goal is to get something working in the wild. AI and AI proponents love this option. It's easy to spot obvious problems, but very unlikely to lead to feedback on structural changes to abstractions and architecture to increase overall _long-term_ velocity.
2. We assume this works, but is it "correct"? This is where long-term code maintainability is created. The quality and effort put into a review like this is obviously far more involved than option 1. People working long term on a code base love this option.
We've been biased towards #1 for a long time, but I feel like we dont have enough people capable of doing #2.
I've worked with some people who only seem to care about 2. as in, they don't try the feature in any way, but come back with comments about "this isn't tested enough" even though it has higher coverage than the codebase's average, and refuse to approve even though they'll never meaningfully review the content. it does seem to be mostly just theater in my experience
exactly, the kind of reviews which have a point are the ones you do before starting work. But that's a design review, not a code review. My team does "commit to master", although I did catch a fair few regressions by looking at the committed code.... but as long as it doesn't go live, who cares?
With a robust enough test suite and a team that does TDD and mob programming, code reviews are pretty much obsolete and a waste of time. Everyone's already involved in the coding process as a mob and the tests catch any regressions.
Yes, I agree, but when the whole team is already part of the development process through mob programming there's no one who hasn't seen the code, hasn't provided feedback/mentorship, and hasn't voiced their stylistic concerns by the time the development is done.
100p agree. Most code reviews devolve to nitpicking anyway. I think it's much more valuable for teammates to review the _design_ and _intent_ rather than the explicit artifacts.
Azure-dev ops explicitly has an "Approve with suggestions", I suspect it was designed for when there are multiple acceptances required, but I sometimes use it anyway.
Whether those comments get read once approved, I don't know.
I see where the author is coming from, but still this feels like a reinvention of commit trailers ("Acked-by", "Reviewed-by",...) especially for these non-blocking and "only appraising" comments.
Commit trailers even have the benefit that they are recorded in the comment, making them discoverable and even allowing statistics, while "human language" comments are not that easy to do statistics on (modulo AI I guess, but that's another topic).
This is one of those articles that I read and realise how different people and process are.
I fully agree with this method. In fact, my surprise mostly stems from the necessity to write this article. So there are places where comments on PRs are avoided as they are considered “unapproved”? Man oh man.
This is more than half of my code reviews (when functionally correct). It works pretty well, when you can trust that the person doing the changes won't accidentally (or otherwise) slip in behavioral changes. Like half just got merged as-is and half of those got a separate cleanup later.
Most people are fine, some routinely abuse it to do much larger changes that should normally imply full review - double check afterward! Your name is still on it, it is still your responsibility.
Just want to comment here. I hate reviewer leave a bunch of nits and stamp the PR. This is ambiguous, are these nits asks or just you opinions? What if I dont address all of them. Also folks need to take rejection lightly - your reviewer wants you to address something, thats really it.
If I've approved the PR, then these are changes I'm asking you to do, but not ordering you to do. You are free to say "no" to my request
> What if I dont address all of them
Then you will have decided that you don't agree with my recommendation and that's OK.
I only ever do this with people I trust - I am trusting you to review each of my nitpicks and make an informed decision if they're necessary or not. Generally I'd like you to reply to the ones you don't do with a reason though.
On Github, there is a way to leave individual comments in the code and in addition give a review a summary.
In addition to hitting the "approve" button, I typically spell it out explicitly in this summary: "Please check my comments and see if anything makes sense to implement."
Often, I also take this opportunity to point out the "one" most valuable change in my opinion.
If the developer of the code doesn't find any of the comments to be applicable/usefuly, they can always go ahead and merge it right away.
I try to tag the line-by-line comments with little labels like [Unimportant] or [Style] so that someone going through them has an idea of their (un)importance without reading the whole thing.
I started doing this a few years ago, it works a treat with non-junior engineers. With the right testing in place, review becomes more about communicating the ideas clearly rather than gatekeeping.
I do this too. I generally skim PRs just to make sure the person isn't doing anything too crazy. I trust my team to write good code. Pulling a branch and testing the code is usually a waste of time unless it's a critical bug or feature.
> I almost always leave a comment on each PR I review, even just observations: “This class is getting big, we might want to consider adding a presenter,” or praise: “Thanks for cleaning this up!”
Things like that I'd much rather leave as comments in the code, rather than dangling as off-hand things in some unrelated PR. Probably no one will see those PR comments unless they go splunking for some reason, but having them right in the code is much more evident to all the people passing through, or even reminding yourself in the future too.
In general I feel like PR reviews are basically reviews happening too late, and if there is a lot of stuff to review and agree upon in the PR, you'll be better off trying to reduce that up front by discussing and noting design decisions before the work even starts. If there is so many unknowns first, do one step to figure out how it can be done, then agree on that, again before starting the actual work.
That leads to PRs basically just being spot-checking, not deep analysis nor even space for nitpicks. If you as a reviewer spots things after the implementation rather in the discussion beforehand, that ends up being on both of you, instead of just the implementer who tried to move along to finish the thing.
You might be approaching PR comments differently than I've seen. When a comment is something to be addressed, it's either put into a new development task (i.e. on something like Jira), or it is completed before the PR merges. I'm not sure that having comments in the code surfaces that information in a useful manner. The code is for the code, not for what the code could be. The comments on what it could be should be handled outside the code at a different abstraction layer.
Agreed. Putting comments in code is good for adding context around the code, but the actual action item needs to be tracked in the same place that all other action items and issues are tracked.
An exception would be for information that doesn't yet qualify as an action item, but could become an action item if someone changes something in the code. Like if removing a conditional check would trigger the need for some other work or a refactor. Then it's good to put it close to the code so anyone touching that code will know they need to make the action items if they go down that route.
> new development task (i.e. on something like Jira), or it is completed before the PR merges.
You've never written an TODO comment? I find them useful.
Perhaps in a small codebase without issue tracking this might be something leverage. It's just not reasonable with 100k+ lines of enterprise grade code.
> In general I feel like PR reviews are basically reviews happening too late, and if there is a lot of stuff to review and agree upon in the PR, you'll be better off trying to reduce that up front by discussing and noting design decisions before the work even starts.
I agree in general and tried to push for a more iterate approach in our team. However, my fear is that this would multiply the effort because it's very easy to discuss many things that do not matter or turn out later to not matter.
It seems it's easier for people to only discuss the important stuff when presented as an actual implementation.
We are talking tendencies here, of course, general design decision must always be done beforehand.
> It seems it's easier for people to only discuss the important stuff when presented as an actual implementation.
LLMs help a lot here, create two prototypes with both designs, compare them together :) Could even evaluate how ergonomic some future potential change is, to see how it holds up in practice.
I used to use pen and paper to essentially do the same, minus having real code and instead just talking concepts, but it does suffer from the issue that some need to be confronted with code in front of them to visualize things better for themselves.
I'm not following this. PRs are the first time your reviewers have seen that change, so there is no opportunity downstream to do the things you're suggesting.
You're essentially suggesting pre-PRs, but it is circular, since those same pre-PRs would have the same criticism.
PRs are about isolated core changes, not feature or system design. They answer how not why.
> You're essentially suggesting pre-PRs, but it us circular, since those same pre-PRs would have the same criticism.
Walking this road to the end you get pair programming.
You get to design committees where everything has to be approved in advance.
Yep, where productivity goes to die and your developers feel no autonomy/trust.
Usually by the time a PR has been submitted it's too late to dig into aspects of the change that come from a poor understanding of the task at hand without throwing out the PR and creating rework.
So it's helpful to shift left on that and discuss how you intend to approach the solution. Especially for people who are new to the codebase or unfamiliar with the language and, thanks to AI, show little interest in learning.
Obviously not for every situation, but time can be saved by talking something through before YOLOing a bad PR.
Yes, it should be cheap to throw out any individual PR and rewrite it from scratch. Your first draft of a problem is almost never the one you want to submit anyway. The actual writing of the code should never be the most complicated step in any individual PR. It should always be the time spent thinking about the problem and the solution space. Sometimes you can do a lot of that work before the ticket, if you're very familiar with the codebase and the problem space, but for most novel problems, you're going to need to have your hands on the problem itself to get your most productive understanding of them.
I'm not saying it's not important to discuss how you intend to approach the solution ahead of time, but I am saying a lot about any non-trivial problem you're solving can only be discovered by attempting to solve it. Put another way: the best code I write is always my second draft at any given ticket.
More micromanaging of your team's tickets and plans is not going to save you from team members who "show little interest in learning". The fact that your team is "YOLOing a bad PR" is the fundamental culture issue, and that's not one you can solve by adding more process.
I don't disagree that a practical spike is a good way to grasp a novel problem (or work with a lack of internal knowledge because it's legacy code) but there is still something to be said for attempting to work things out in the abstract too, and not necessarily by adding process, but by redeveloping that internal knowledge and getting familiar with the business domain.
In a greenfield project I will have a lot of patience for a team that doesn't grasp the problem space too well yet, and needs to feel around it by experimenting and prototyping. You have to encourage that or you might not even be building anything innovative.
For the longer term legacy project then the team can't really afford to have people going down rabbit holes and it's more beneficial to approach things in the abstract and reduce the problem as much as possible. Especially with junior or mid-level engineers who can see an old codebase as a goldmine for refactoring if left unattended.
As for the fundamental culture issue... maybe. AI increases the frequency of low quality PRs and puts a bigger burden on the reviewer. I can live with this in the short term if people take lessons from it and keep building up their own skillset. I feel this issue is not unique to my team and LLM-driven development is still novel enough that we're all figuring out the best way to tackle it.
I'm not sure what approach you're suggesting?
Asking a more junior developer or someone who "show little interest in learning" to discuss their approach with you before they've spent too much time on the problem, especially if you expect them to take the wrong approach seems like the right way to do things.
Throwing out a PR of someone who doesn't expect it would be quite unpleasant, especially coming from someone more senior.
This is how I try to approach it. I don't think it's a new thing for a new hire to come in hot and try to figure things out themselves rather than spending time with the team. Or getting lost down rabbit holes.
> PRs are the first time your reviewers have seen that change, so there is no opportunity downstream to do the things you're suggesting.
Yes, but I'm arguing for that it shouldn't be the first time they hear about that this change is planned and happening, and their input should have taken into account before the PR is even opened, either upfront/before or early in development. This eliminates so many of the typical PR reviews/comments.
Figure out where you are going before you start going there, instead of trying to course correct after people already walked places.
I think it's a fine line to walk. At my job what we do is discuss any complex implementation or risky change or blockers in the dev eng meeting. For smaller stuff, or more straightforward solutions, we don't bring it up. If you make it a hard rule to first discuss all tickets, it just seems draconian.
Code review is specifically for code quality, more lower level stuff.
When I review a PR, I don't add comments to the code. If I think something needs to be commented: I comment on the PR and reject the PR.
If I think --as is discussed in the article-- that the comment "would be nice, but is not strictly necessary", then I comment on this on the PR and approve the PR.
> If you as a reviewer spots things after the implementation rather in the discussion beforehand, that ends up being on both of you, instead of just the implementer who tried to move along to finish the thing
This is accurate, but it's still an important check in the communication loop. It's not all that uncommon for two engineers to discuss a problem, and leave the discussion with completely different mental models of the solution.
This one resonates, because it's largely how I comment on PRs too!
One thing not mentioned is how important it is to acknowledge the comments too. People are taking their time to review your PR, and not even giving a reaction will make the commenter question whether or not it was even received. I'm not looking to throw my thoughts out into the aether. That's what microblogging platforms are for!
I can't tell you how many times I got no response/acknowledgement on a comment that actually surfaced something critical. I haven't been keeping track, but I think my comments could have prevented dozens of outages at this point. It's quite exhausting. In my own experience, the worst offenders of this are senior devs.
> Why approve, if I’ve left comments that I think are worth implementing? > > Because I trust my team. I know that my comments will be considered, and if they’re useful, implemented.
I do this a lot too. It's critical that PR authors don't burn that trust either. If they make substantial changes that warrant another review, I hope they do request it. Too many times in my career have colleagues just went ahead and made bad changes after my approval that I would have easily caught, merge, and things go
High trust, high alignment environments move so fast, and you know when you're in one and know when you have your colleagues' trust. It feels really good!
Isn’t a change to the code that incorporates your suggestions enough acknowledgment? Presumably the code change would be required to get your approval?
Of course! I didn't state it explicitly, but what I was referring to were comments that went completely unacknowledged in any way, be it explicit or implicit. No changes. No reaction. No "resolve comment". Radio silence.
As mentioned, I've experienced it too many times where not addressing a question/concern I put on the pull request led to outages that could have been avoided. I think it's typically a certain personality. It's not a common occurrence, but I have experienced it.
While one of the best things about PRs can be raising the floor on quality, I think the another powerful factor is forcing an opportunity to talk about the code. This article even pre-empts this confusion by acknowledging that commenting on a PR while approving it is "weird". This implies that many people don't understand the indirect benefits of the PR and think the process is all about approval and raising the quality floor.
I think raising the floor, gives diminishing returns once someone is used to the team and the code base, but the conversation always remains relevant. Sometimes teams that resist things like PRs (e.g. "they just slow us down") are actually teams that are having those conversions elsewhere (in-person, on slack, during standup or sprint planning, etc).
Totally agreed, but it always comes back to managing the average over the best. It is a lot more effective to enforce a culture of approvals than a culture of collaboration. Managers should be striving for the later, but not every team is built the same way and many in leadership just need assurance that the bare minimum is getting done.
> Additionally, some repos can be configured to automatically merge PRs when all requirements are met, one of which might be your approval.
If anyone at GitHub is reading this, I’d love a fourth checkbox in the “leave a review” modal that is “Approve but disable auto merge” (alongside Comment/Approve/Request changes)! Even just surfacing “this PR has auto merge enabled” near the Approve button would be great.
You might try adding this branch protection rule to require conversation resolution: https://docs.github.com/en/repositories/configuring-branches...
This feels like a suboptimal solution to me because I personally like to keep comments in "unresolved" state so that they remain visible and other folks can weigh in on them if they want, but in a way that doesn't block the PR. Basically I wish that GitHub would either separate the "collapsed" and "resolved" concepts, or add this "approve without merging" button.
I do the same as the author, and for my comments I make it clear what is non-blocking/blocking with Conventional Comments: https://conventionalcomments.org
To make it easier for myself to leave the CC tags ("nit(non-blocking): ", etc), I use the macOS text expander in System Settings and created mnemonics to easily insert them.
Example: If I type "+pnn", that maps to:
+p = pull request comment n = nit n = non-blocking
Example 2: If I type "+pc", that maps to:
+p = pull request comment c = chore
(and it's blocking because I didn't type "+pcn", the non-blocking version).
I'm a big believer in this workflow and generally agree with all the guidelines about when to do and not do this.
Code review has value, but I don't think we are always honest about the costs. At most places I've worked, there has been an informal understanding that code should be reviewed within 24 hours or so, but there is a world of difference between getting a review within 2 hours and 23 hours.
If you have to go back and forth a second time, it's so much more likely that the approval comes much later due to straddling end-of-day in someone's timezone.
Tangentially, if you are designing policies for code review at your org, try to think both about minimum requirements (e.g. PRs should get a first look within 24 hours) and typical requirements (e.g. most PRs should get reviewed within 2-3 hours). What typically happens is what determines overall velocity for your org, so it's much more important than whatever strict SLA policy you have. These are also fixable problems if you have targeted conversations with people. E.g. "I noticed X, Y, Z times, you had unreviewed PRs at the end of the day. Can you set aside some time to review code before EOD? Or try installing PR reminder?"
I do the same!
But PR discussions about lintable style issues always surprise me. The ideal solution is to add a rule in the linter. But when the team won't agree on the rule, and is open to multiple styles, the author should decide, simple! Had a team mate recently who'd block PRs over T[] vs Array<T>, forcing people to stick with Array<T> for simple types like number[] even though TypeScript's own docs and tooling push T[].
Linters often don’t provide the constraints you desire. While they’re a great (arguably essential) tool, they often are not sufficient.
When a project has well-established patterns, part of the job is to just follow the pattern, whether you like it or not, until you find a reason to 1) change the pattern everywhere, or; 2) make an exception to the pattern with intention.
It's quite simple to write a rule, e.g. for the example I shared https://typescript-eslint.io/rules/array-type. But I've never seen any reputable project arguing against T[] for simple types. In my opinion, that says a lot about the reviewer, not the contributor. If a linter rule doesn't exist, you should argue against the idiomatic convention.
I’m not arguing any particular rule… But what linter are you assuming I’m running? You can’t always write your own rules.
Write a lint rule. It's often quite easy.
What linter are you assuming I’m running? You can’t always write your own rules.
Yeah, I think all style comments should be handled by either a linter/formatter or be written in a style guide. Everything else is up to personal preference.
Then, when a style comment comes up in a PR, the answer is "Oh, do you think we should add that to our style guide? If so, let's discuss that in slack. Until then, though, that's not blocking."
This is exactly the pedantry I try to avoid in my teams.
The point of a review shouldn’t be to make sure it’s exactly how the reviewer would do it. Is it safe? Does the author understand the area? Is there evidence the code has been exercised? Does it follow the major conventions?
Beyond that, I try very hard to build a culture of approvals vs nitpicking, and let the linters enforce the rest.
Same. I've said my peace, but I don't let that get in the way of what works.
Notably, prevent github merges if there are unresolved comments, so you know they glanced over them before collapsing it.
This is me, I think it comes from a personality of wanting to make sure I say something but generally trusting that the author has more context than I do. I float questions and nits but generally will approve if I don't see anything glaring. If I spot anything where I think "we should definitely go another direction here" I ask for changes instead of approving and make them super clear.
We do this too. In my team, my rule is: if it’s better than what’s on master, you approve and merge.
There’s no use making the customer wait for your questions, code style suggestions etc to be addressed.
Even if you request changes, you leave all your comments and make explicit which are the blocking ones and which can be addressed in the future.
> In my team, my rule is: if it’s better than what’s on master, you approve and merge.
This causes unnecessary code changes later on, code changes mean new code, new code has bugs. The team should try to get it close to perfect on the first try instead. They won't, but that should be everyone's target. If that sounds impossible, then the PR was to big.
Totally agree, and I go even farther: I consider that the PR author is totally responsible of merging their PR or not, and so no comment is blocking. I can make a comment I consider blocking, but there might be very good reasons that I can't see to still merge the PR. Fixes can come in a follow-up PR. As long as everyone takes responsibility for their work, it is a well-working system.
I like doing this as well.
The 'auto merge on approval flag' PR authors can flip on GitHub breaks this flow though, as it will just merge as soon as you hit approve.
I also follow this approach. I just flip the flag on the PR I’m reviewing to off before submitting my approval.
We also have most of our repos set to block if unresolved comments. I think it’s a flag on branch protection rules
The way my company works, it doesn’t break it. You approve and merge, then any suggestions can be implemented in a next PR
That way the average customer doesn’t need to wait for your code style change or edge case fix
I woke up one day and the industry is gating all code changes behind PRs w/ approvals. Anyone still able to commit to main without approval from someone else? I’m even seeing companies require multiple approvals on every PR. It’s insanity.
I’m skeptical. Software is as buggy as it ever was. I come across teams shipping terrible quality software, where every line change was approved and reviewed. I come across teams that require every change have an approval, but don’t require 100% test coverage. I’m seeing senior engineers have to get approval from juniors for a copy change.
It’s theatre. It’s bad management. It’s a cargo cult. It isn’t actually driving code quality.
Code review is one thing. Code review is good. But requiring every change have an approval is something else.
Where I work I can and am still allowed to push to mainline directly, as are all other developers on the team.
Some new colleagues started with PRs and since then it has been a slow move towards using PRs more and more (but still not mandatory).
As of now some colleagues don't typically do PRs and push directly (minority), some decide on a case-by-case basis (I am among those, among a small majority in our team) and some have been using PRs for each and every change from day one (also a minority).
The criticisms by the opponents of PRs are as follows:
1. People relying on PRs too much causes them to propose code which is not production quality as of the PR whereas before (i.e. "no PRs") if your code was not correct, you or someone else either noticed shortly after (checking the tests on CI etc.) or it would make it into the release. I don't believe this to be generally true but sometimes thought similar when reading some PRs. The frequency of large bugs being found by code review in the PR has gone down in recent times which leads me to believe that the colleagues have adjusted their development style to come up with good quality solutions in the first attempt already in most cases.
2. Code review is hard to do in many cases. In our team this is typically resolved by going over a PR together for the “hard cases” (e.g. in a videoconference).
I think PRs are mostly worth the effort because any bug that can be avoided before the release saves a lot of downstream effort (e.g. support) and the effect of building a shared knowledge about the code page is valuable (although hard to quantify).
There's two kinds of reviews in my experience:
1. Does it work? Then ship it. This is great for early on, high-velocity where the goal is to get something working in the wild. AI and AI proponents love this option. It's easy to spot obvious problems, but very unlikely to lead to feedback on structural changes to abstractions and architecture to increase overall _long-term_ velocity.
2. We assume this works, but is it "correct"? This is where long-term code maintainability is created. The quality and effort put into a review like this is obviously far more involved than option 1. People working long term on a code base love this option.
We've been biased towards #1 for a long time, but I feel like we dont have enough people capable of doing #2.
I've worked with some people who only seem to care about 2. as in, they don't try the feature in any way, but come back with comments about "this isn't tested enough" even though it has higher coverage than the codebase's average, and refuse to approve even though they'll never meaningfully review the content. it does seem to be mostly just theater in my experience
exactly, the kind of reviews which have a point are the ones you do before starting work. But that's a design review, not a code review. My team does "commit to master", although I did catch a fair few regressions by looking at the committed code.... but as long as it doesn't go live, who cares?
With a robust enough test suite and a team that does TDD and mob programming, code reviews are pretty much obsolete and a waste of time. Everyone's already involved in the coding process as a mob and the tests catch any regressions.
There's two purposes to review. One is to enforce ownership, and the other is to provide mentorship.
Even if you don't have ownership, mentoring is still useful.
I get asked for review by teammembers on an area I have expertise in.
Their solution might work but cause problems later. With review, I can knowledge transfer my lived experience so they don't suffer like I did.
The third purpose to review is stylistic nitpicking and formatting as a simulacrum of actual work. This is useless and turns people off from review.
Yes, I agree, but when the whole team is already part of the development process through mob programming there's no one who hasn't seen the code, hasn't provided feedback/mentorship, and hasn't voiced their stylistic concerns by the time the development is done.
If your team is 5 people then sure, it's fine.
From my experience in a massive shared repo with unclear ownership...
100p agree. Most code reviews devolve to nitpicking anyway. I think it's much more valuable for teammates to review the _design_ and _intent_ rather than the explicit artifacts.
Azure-dev ops explicitly has an "Approve with suggestions", I suspect it was designed for when there are multiple acceptances required, but I sometimes use it anyway.
Whether those comments get read once approved, I don't know.
I see where the author is coming from, but still this feels like a reinvention of commit trailers ("Acked-by", "Reviewed-by",...) especially for these non-blocking and "only appraising" comments. Commit trailers even have the benefit that they are recorded in the comment, making them discoverable and even allowing statistics, while "human language" comments are not that easy to do statistics on (modulo AI I guess, but that's another topic).
This is one of those articles that I read and realise how different people and process are.
I fully agree with this method. In fact, my surprise mostly stems from the necessity to write this article. So there are places where comments on PRs are avoided as they are considered “unapproved”? Man oh man.
This is more than half of my code reviews (when functionally correct). It works pretty well, when you can trust that the person doing the changes won't accidentally (or otherwise) slip in behavioral changes. Like half just got merged as-is and half of those got a separate cleanup later.
Most people are fine, some routinely abuse it to do much larger changes that should normally imply full review - double check afterward! Your name is still on it, it is still your responsibility.
Just use blocking/non-blocking.
The issue with nitpicks: different reviewers have different definitions of what a "nit" is.
Just want to comment here. I hate reviewer leave a bunch of nits and stamp the PR. This is ambiguous, are these nits asks or just you opinions? What if I dont address all of them. Also folks need to take rejection lightly - your reviewer wants you to address something, thats really it.
Hi, I do this.
> are these nits asks or just you opinions?
If I've approved the PR, then these are changes I'm asking you to do, but not ordering you to do. You are free to say "no" to my request
> What if I dont address all of them
Then you will have decided that you don't agree with my recommendation and that's OK.
I only ever do this with people I trust - I am trusting you to review each of my nitpicks and make an informed decision if they're necessary or not. Generally I'd like you to reply to the ones you don't do with a reason though.
On Github, there is a way to leave individual comments in the code and in addition give a review a summary.
In addition to hitting the "approve" button, I typically spell it out explicitly in this summary: "Please check my comments and see if anything makes sense to implement."
Often, I also take this opportunity to point out the "one" most valuable change in my opinion.
If the developer of the code doesn't find any of the comments to be applicable/usefuly, they can always go ahead and merge it right away.
I try to tag the line-by-line comments with little labels like [Unimportant] or [Style] so that someone going through them has an idea of their (un)importance without reading the whole thing.
I started doing this a few years ago, it works a treat with non-junior engineers. With the right testing in place, review becomes more about communicating the ideas clearly rather than gatekeeping.
Does anyone not do this? It would be miserable working at a place where nits were blockers and required re-review.
This advice can also be applied to PhD thesis examinations and paper reviews.
I do this too. I generally skim PRs just to make sure the person isn't doing anything too crazy. I trust my team to write good code. Pulling a branch and testing the code is usually a waste of time unless it's a critical bug or feature.
I approve of it.
I have now commented on it.
[flagged]