These pieces of advice are useful. However, they don't touch the bottleneck: mental health. And no, it is not "like any other demanding job". A PhD hits on two fronts - one is "all or nothing". If you spend years and still haven't submitted your dissertation, it is a failure. The other is its tie to one's identity. You put sweat, blood, and tears into your research, only to be rejected at a journal or conference because the result is "technically correct but not significant enough". Sure, there are similar parts in other careers - from talking with people, it works a bit similarly in medicine (when it comes to "all or nothing") and art (when it comes to this identity).
If people fail, it is mostly because they burn out. If they succeed, it is not unlikely that they will need to heal their burnout wounds anyway.
I am sure Karpathy's experience is different. But most people starting their PhDs are not Karpathy.
The peer review paper requirement puts you in a situation where if your topic of research happen to not be interesting for the reviewers (that you have no control over), you can be a talented student that worked very hard and still fail due to being out of time after multiple successive rejections.
Your supervisor may not understand this until it’s too late, and you may not have the ability to judge your adviser's ability to do so until you are committed.
The main problem is that you were raised in a school system where if you show up, study and do your assignments you are pretty much guaranteed to succeed sooner or later. A PhD is not like that.
The one piece of advice I give new PhD students is to maintain a list of your references for a bibliography ahead of time. For every paper you read, copy the citation in BibTeX format and write a couple of sentences to remind yourself what the paper was about. Do this for every source, even if it doesn't seem important at the time.
Use zotero and betterbibtex. By all means type a comment so you know which ideas came from where but I'm a big advocate of taking notes by hand when you really want to understand something, as opposed to reminding your future self about something you already understand.
Not within a PhD, but as a side project I work on a research project on wikiversity about grammatical gender in French. It does reference a bunch of books and academic works, like probably a hundred I guess. The most tedious work though is to check which nouns are used only in a single gender of do have some epicenic or specific inflection used in the wild and giving a reference that attest that when it's not already so consensual that most general public dictionary would already document the fact. For that the research refers to thousand of webpages. I'm glad that most of the time I just need to drop the DOI, ISBN, or page URL and MediaWiki will handle the filing of the most relevant fields. That's not perfect, it generates the output with many different models currently (some don't have an excerpt field), and some required fields might be left blank, url to pdf won't work, and so on. But all in all it make the process of taking note of the reference quick and not going too much in my way. Creating a structured database out of it can certainly be done later.
I have had some fun exhuming my old LaTeX skills and assembling a BibTeX bibliography from which I automatically extract the right entries presented in whichever style is needed for a given paper and for my own (HTML) site. I even publish the collection in Zenodo in case useful to others. I use the 'annote' field for the reminder you suggest.
The lack of good tools to have good research notes with good search is kind of mind-boggling. I have reverted to having a website for myself, a private one that I run on my machine, using mkdocs which comes close to what I would want.
Zotero and AI have this covered now. If there's one thing AI is good at it's summarising crappy formatted papers. Never understood the 2 and 3 column thing. Horrendous way to format something.
The comments you write in to Zotero are not what paper is about - abstract covers this well enough - it’s about what you found interesting or useful about the paper.
2-column format has narrower columns, which means that your eyes move more vertically than horizontally while reading it. That is considered conducive to “skimming” long texts if you’re a “speed reader”.
Do you mean that you’re using AI as a search engine for your local bibliography? I haven’t seen any AI plugins for Zotero.
I feel like that's true when the font is insanely small, which I guess was good when people would print entire proceedings.
Reading two column super small font on a computer is super annoying though tbh.
I'm severely dyslexic and the columns are a massive hindrance for me, and I also cannot skim read due to this and meares irlen. So my dislike is not universal applicable, just personal experience.
On the zotero front there a bunch of AI plugins. But I've not used them. But yes the premise is that your can speak and ask your library questions. Some are set up differently though. Personally I can fire a paper into an llm and get a good idea of the content immediately and then ask questions about it. It's more interactive and allows me to get a better idea of it prior to reading it.
LLMs make too many mistakes when summarizing papers in their current state, I would never trust it to summarize a whole paper at the moment.
I only use it on a sentence or paragraph basis, otherwise it misses the point 90% of the time.
I would strongly advise against this use for the moment.
The important part of reading a paper is not only to extract general rules, but to build your own internal model. Without it you cannot effectively do research. The main interesting points are often in the subtleties of the details deep in the paper.
Internal tought that come easily to mind when I read :
- 'oh they used that equation, but that could be also be interpreted totally differently, what happens if we change point of view, does it makes sense from this other perspective'
- 'I see they claim to achieve better results than sota, but actually, they compared with other methods that are not solving exactly the same problem, what shortcut or changes did they had to do to obtain a fair comparison, is it a fair comparison, can I trust those numbers? '
- 'oh, the authors didn't realize that they solved this other problem, or did they realize but there was a block somewhere preventing it?'
- 'I like this trick to achieve that result, but at the same time, it will prevent to solve a whole class of other problems, so their method will not work on those cases'
...
Also, notice that a paper IS a summary of multiple months/years of work, and researchers summarize it already to the maximum to stay within the page limit, by summarizing a summary you will always miss many things.
Fair points. And likely why I'm not suited to academia too. I've just never really groked the practice. I've obviously only experience from bachelor's and masters but it always seems that you have an idea and the research is finding papers to back it up, and then some that might not. The work you do doesn't really matter as it seems secondary to the nonsense around "the literature".
“How to get into a top PhD program: get ~3 famous professors to write letters saying you’re in the top 5 students they’ve ever worked with.”
I feel like this particular advice applies to a very small subset of people. If I’d had professors telling me that I certainly would have considered doing a PhD!
Karpathy is an interesting case of PhD gone industry and he mentions this topic in the article. In my field of computational social science it is sadly very taboo to happily leave the academy. Yet, they don’t do much to make it more appealing. My biggest win was to find a group of people outside of my research group that I liked collaborating with. Research is more fun as a team sport.
Interesting. A couple of questions:
- How young are the kids?
- How do they behave, especially with essentials like eating and sleeping habits?
- Could you carve out a morning and/or evening routine for yourself?
- How much outside help could you rely on (grandparents nearby, lovely neighbours...)?
I did a bachelor degree part time later in life around work and family life. I'm doing a masters full time around work and family life. My experience with academia so far have put me off further study. I really don't get the research thing, and the whole experience seems like bullshit to me. Out of all my experiences doing these things the best has been on the taught modules, that I enjoyed and I didn't feel were out of date, the worse has been the dissertations where you're doing "research". Think of a project off the top of your head and "research" it. Nonsense.
One thing that's not mentioned here: if you don't come from a top university, you have close-to-zero chances to have that kind of experience in your phd. If you're not incredibly picking some exceptionally relevant project soon enough, your career path after the phd will not be exactly the smooth sailing the author describes.
Loved this article. I'd add a few things I wish someone had told me when I was starting my PhD: 1) Maximize variance, but know when to stop. Karpathy's point is great. Explore early, say yes to different things. But at some point you need to pick a direction and commit. Too much variance and you end up with nothing solid. 2) Consider smaller labs. Big famous groups are tempting, but in a small group of 3-5 people your adviser actually knows your work and gives you real feedback. In large labs you can easily become invisible. 3) Collaborate outside your lab early. Don't wait, reach out to people at other universities working on related problems. Different groups think differently and that's where good ideas come from. 4) Visit other universities. Even a few weeks at another group forces you to explain your work to people with different assumptions. It's one of the most useful things you can do during a PhD. 5)Learn to write good, structured, reproducible and maintainable code. One of the things I regret I didn't, and many working hours were wasted.
One angle that's missing from this discussion: the cost asymmetry. The effort to do this well is disproportionate to the effort to do it poorly, which means most implementations in the wild are mediocre. That creates a weird market dynamic where quality becomes the differentiator by default — not because it's hard to build, but because most people stop at 'good enough' too early.
I can imagine that this will be similar to the "Emacs/Vim in the AI age" article - it will just be considered to matter less in the AI age. Why spend 3-5 years of your life with a sometimes frustrating experience to obtain this PhD degree if you have powerful models at your disposal that will just be able to solve everything for you? (Similar to why learn Elisp/VimScript/...) Especially considering the current trajectory, expecting where things will be in 5 or 15 years. It will just feel less and less appealing to get an in-depth education, especially a formal one.
Which is quite ironic, considering who wrote the article.
LLMs fall victim to "garbage in, garbage out." Claude can solve open problems if you know what you're doing, but it can also incorrectly convince you it's right if you don't know what you're doing.
A PhD teaches you how to think, how to learn, and how to question the world. That's a vital set of skills no matter what tool exists.
Why spend your life doing anything at all? I'm biased on the topic since im writing up atm, but it was, if nothing else, a very itnerseting way to spend 4 years of my life.
I find it very fulfilling to do a PhD and did so myself. More people should. What I mean is that I'm expecting the general view on it to evolve as described.
These pieces of advice are useful. However, they don't touch the bottleneck: mental health. And no, it is not "like any other demanding job". A PhD hits on two fronts - one is "all or nothing". If you spend years and still haven't submitted your dissertation, it is a failure. The other is its tie to one's identity. You put sweat, blood, and tears into your research, only to be rejected at a journal or conference because the result is "technically correct but not significant enough". Sure, there are similar parts in other careers - from talking with people, it works a bit similarly in medicine (when it comes to "all or nothing") and art (when it comes to this identity).
If people fail, it is mostly because they burn out. If they succeed, it is not unlikely that they will need to heal their burnout wounds anyway.
I am sure Karpathy's experience is different. But most people starting their PhDs are not Karpathy.
See also "The Lord of the Rings: an allegory of the PhD?" http://danny.oz.au/danny/humour/phd_lotr.html
Sure you may survive. But even if all goes well, you succeed, there will be a void in you after the quest.
The peer review paper requirement puts you in a situation where if your topic of research happen to not be interesting for the reviewers (that you have no control over), you can be a talented student that worked very hard and still fail due to being out of time after multiple successive rejections.
Your supervisor may not understand this until it’s too late, and you may not have the ability to judge your adviser's ability to do so until you are committed.
The main problem is that you were raised in a school system where if you show up, study and do your assignments you are pretty much guaranteed to succeed sooner or later. A PhD is not like that.
The one piece of advice I give new PhD students is to maintain a list of your references for a bibliography ahead of time. For every paper you read, copy the citation in BibTeX format and write a couple of sentences to remind yourself what the paper was about. Do this for every source, even if it doesn't seem important at the time.
Use zotero and betterbibtex. By all means type a comment so you know which ideas came from where but I'm a big advocate of taking notes by hand when you really want to understand something, as opposed to reminding your future self about something you already understand.
There's also better notes too for this.
Not within a PhD, but as a side project I work on a research project on wikiversity about grammatical gender in French. It does reference a bunch of books and academic works, like probably a hundred I guess. The most tedious work though is to check which nouns are used only in a single gender of do have some epicenic or specific inflection used in the wild and giving a reference that attest that when it's not already so consensual that most general public dictionary would already document the fact. For that the research refers to thousand of webpages. I'm glad that most of the time I just need to drop the DOI, ISBN, or page URL and MediaWiki will handle the filing of the most relevant fields. That's not perfect, it generates the output with many different models currently (some don't have an excerpt field), and some required fields might be left blank, url to pdf won't work, and so on. But all in all it make the process of taking note of the reference quick and not going too much in my way. Creating a structured database out of it can certainly be done later.
I have had some fun exhuming my old LaTeX skills and assembling a BibTeX bibliography from which I automatically extract the right entries presented in whichever style is needed for a given paper and for my own (HTML) site. I even publish the collection in Zenodo in case useful to others. I use the 'annote' field for the reminder you suggest.
The lack of good tools to have good research notes with good search is kind of mind-boggling. I have reverted to having a website for myself, a private one that I run on my machine, using mkdocs which comes close to what I would want.
Zotero and AI have this covered now. If there's one thing AI is good at it's summarising crappy formatted papers. Never understood the 2 and 3 column thing. Horrendous way to format something.
The comments you write in to Zotero are not what paper is about - abstract covers this well enough - it’s about what you found interesting or useful about the paper.
2-column format has narrower columns, which means that your eyes move more vertically than horizontally while reading it. That is considered conducive to “skimming” long texts if you’re a “speed reader”.
Do you mean that you’re using AI as a search engine for your local bibliography? I haven’t seen any AI plugins for Zotero.
I feel like that's true when the font is insanely small, which I guess was good when people would print entire proceedings. Reading two column super small font on a computer is super annoying though tbh.
I'm severely dyslexic and the columns are a massive hindrance for me, and I also cannot skim read due to this and meares irlen. So my dislike is not universal applicable, just personal experience.
On the zotero front there a bunch of AI plugins. But I've not used them. But yes the premise is that your can speak and ask your library questions. Some are set up differently though. Personally I can fire a paper into an llm and get a good idea of the content immediately and then ask questions about it. It's more interactive and allows me to get a better idea of it prior to reading it.
LLMs make too many mistakes when summarizing papers in their current state, I would never trust it to summarize a whole paper at the moment.
I only use it on a sentence or paragraph basis, otherwise it misses the point 90% of the time.
I would strongly advise against this use for the moment. The important part of reading a paper is not only to extract general rules, but to build your own internal model. Without it you cannot effectively do research. The main interesting points are often in the subtleties of the details deep in the paper.
Internal tought that come easily to mind when I read :
- 'oh they used that equation, but that could be also be interpreted totally differently, what happens if we change point of view, does it makes sense from this other perspective'
- 'I see they claim to achieve better results than sota, but actually, they compared with other methods that are not solving exactly the same problem, what shortcut or changes did they had to do to obtain a fair comparison, is it a fair comparison, can I trust those numbers? '
- 'oh, the authors didn't realize that they solved this other problem, or did they realize but there was a block somewhere preventing it?'
- 'I like this trick to achieve that result, but at the same time, it will prevent to solve a whole class of other problems, so their method will not work on those cases'
...
Also, notice that a paper IS a summary of multiple months/years of work, and researchers summarize it already to the maximum to stay within the page limit, by summarizing a summary you will always miss many things.
Fair points. And likely why I'm not suited to academia too. I've just never really groked the practice. I've obviously only experience from bachelor's and masters but it always seems that you have an idea and the research is finding papers to back it up, and then some that might not. The work you do doesn't really matter as it seems secondary to the nonsense around "the literature".
In what, that does not align at all with my experience in the sciences (where the point is the novelty, not summarise the literature)?
“How to get into a top PhD program: get ~3 famous professors to write letters saying you’re in the top 5 students they’ve ever worked with.”
I feel like this particular advice applies to a very small subset of people. If I’d had professors telling me that I certainly would have considered doing a PhD!
Karpathy is an interesting case of PhD gone industry and he mentions this topic in the article. In my field of computational social science it is sadly very taboo to happily leave the academy. Yet, they don’t do much to make it more appealing. My biggest win was to find a group of people outside of my research group that I liked collaborating with. Research is more fun as a team sport.
I finished a PhD while working full time with 3 young kids. Feel free to reach out if you've been interested and I can share my experience with you.
How did you keep the motivation up?
(I tried doing a PhD while working full time, and quit the idea after 3 years.)
Interesting. A couple of questions: - How young are the kids? - How do they behave, especially with essentials like eating and sleeping habits? - Could you carve out a morning and/or evening routine for yourself? - How much outside help could you rely on (grandparents nearby, lovely neighbours...)?
I did a bachelor degree part time later in life around work and family life. I'm doing a masters full time around work and family life. My experience with academia so far have put me off further study. I really don't get the research thing, and the whole experience seems like bullshit to me. Out of all my experiences doing these things the best has been on the taught modules, that I enjoyed and I didn't feel were out of date, the worse has been the dissertations where you're doing "research". Think of a project off the top of your head and "research" it. Nonsense.
You are right, sir, you should stay away from research. Don't worry, there are others that will handle it.
Why don't we assign grad students to PhD courses the way NFL draft works?
Let directional universities pick first and Ivies (and other prestigious universities) pick last.
One thing that's not mentioned here: if you don't come from a top university, you have close-to-zero chances to have that kind of experience in your phd. If you're not incredibly picking some exceptionally relevant project soon enough, your career path after the phd will not be exactly the smooth sailing the author describes.
Loved this article. I'd add a few things I wish someone had told me when I was starting my PhD: 1) Maximize variance, but know when to stop. Karpathy's point is great. Explore early, say yes to different things. But at some point you need to pick a direction and commit. Too much variance and you end up with nothing solid. 2) Consider smaller labs. Big famous groups are tempting, but in a small group of 3-5 people your adviser actually knows your work and gives you real feedback. In large labs you can easily become invisible. 3) Collaborate outside your lab early. Don't wait, reach out to people at other universities working on related problems. Different groups think differently and that's where good ideas come from. 4) Visit other universities. Even a few weeks at another group forces you to explain your work to people with different assumptions. It's one of the most useful things you can do during a PhD. 5)Learn to write good, structured, reproducible and maintainable code. One of the things I regret I didn't, and many working hours were wasted.
Good luck to anyone starting out.
Good to see this again resurface !
One angle that's missing from this discussion: the cost asymmetry. The effort to do this well is disproportionate to the effort to do it poorly, which means most implementations in the wild are mediocre. That creates a weird market dynamic where quality becomes the differentiator by default — not because it's hard to build, but because most people stop at 'good enough' too early.
I can imagine that this will be similar to the "Emacs/Vim in the AI age" article - it will just be considered to matter less in the AI age. Why spend 3-5 years of your life with a sometimes frustrating experience to obtain this PhD degree if you have powerful models at your disposal that will just be able to solve everything for you? (Similar to why learn Elisp/VimScript/...) Especially considering the current trajectory, expecting where things will be in 5 or 15 years. It will just feel less and less appealing to get an in-depth education, especially a formal one.
Which is quite ironic, considering who wrote the article.
LLMs fall victim to "garbage in, garbage out." Claude can solve open problems if you know what you're doing, but it can also incorrectly convince you it's right if you don't know what you're doing.
A PhD teaches you how to think, how to learn, and how to question the world. That's a vital set of skills no matter what tool exists.
It seems your question largely boils down to: “why do anything when AI could do it instead?”
I think there are many answers to this, not the least of which is that AI can’t really do it instead.
Why spend your life doing anything at all? I'm biased on the topic since im writing up atm, but it was, if nothing else, a very itnerseting way to spend 4 years of my life.
People seem to get my comment wrong.
I find it very fulfilling to do a PhD and did so myself. More people should. What I mean is that I'm expecting the general view on it to evolve as described.
Doing hard things has consistently made me more generally (not only in the narrow hard thing) competent and comfortable with myself.
Why go to the gym if you don't need physical strength? One needs to do something to not degenerate into a miserable state.
Models can solve the problem, but they can't tell you if the problem was worth solving in the first place.