There are many articles extolling the virtues of Lisp. I would like to see some articles that have a level-headed criticisms or critique of Lisp, it's ideas and it's place in the ecosystem of languages.
Articles like this, and the PG articles it references, amount to "if you know, you know". I understand the appeal and I understand the explicit and implicit arguments this article is making.
Computer programming has matured quite a bit in the past 60 years. I would like to see more articles that are more considered in their examination.
Programming is in tension between the Light Side and the Dark Side.
The Light Side is about preventing the programmer from making mistakes: Get rid of go-tos! Add static types! Do not allow a bug to be expressible. The Dark Side is about giving power to the programmer: Macros? Obviously. Operator overloading? Self-modifying code? Multi-line reg-exps? Go to town!
The Light Side knows programmers are flawed and imposes constraints. The Dark Side trusts programmers with power. Neither side is correct all of the time, and a good programmer learns both.
Lisp is interesting in that it is clearly Dark Side programming (the programmer can do anything) but it's still admired by Light Side programmers. Maybe there's something about the simplicity of the language that makes it seem platonic--almost incorruptible. Or maybe Lisp is so pure that it embodies both Light Side and Dark Side, like a god that spawned the programming universe.
The website seems to have a bug with syntax highlighting. Pieces of code included in the post text are black, you can still see the actual text if you select it with your mouse. Same bug on Chrome desktop and on Safari on iPad
There are some truly powerful and unique things about Lisps, but I wish articles like this would stop including REPLs and hot-reloading. The former have been table stakes for interpreted languages (and some compiled ones!) for years, and the latter is neither unique nor particularly widely used (hot reloads have to tangle with state and patching, so resetting the world for ease of reasoning is considered a best practice for a reason).
> The former have been table stakes for interpreted languages
I used to think so too back in the day when I was getting into Clojure. It was much later when I realized that when Lisp people talk about the REPL they’re usually talking not so much about the interactive CLI where you can evaluate commands easily but more so the ability to connect your program to a live session where you can quickly evaluate forms within your text editor and in the context of your running application, which enables much more interactive development than in other interpreted languages.
I do think it is worth highlighting how many advanced parts of hot-reloading have already been covered in Common Lisp. Same with highlighting how the REPL is largely not used to directly type into, but is instead a very powerful interface for tools to interact with a running image.
But, again agreed that simply these existing are not that notable today.
I must admit - I still don't understand macros. I get that they're code that's generated at compile time. But I don't understand how that's different than a function which evaluates other functions. I guess the latter would actually be evaluated at runtime? I think I get it conceptually but I'm not sure I have the muscle memory to reach for them. Anybody here have an "ah hah!" Moment with macros?
If you haven't read it, I'd suggest taking a look at Paul Graham's book On Lisp [0]. He says better, and with more examples than I'd provide in a comment block, what I'd write on the subject. Jump to chapter 8 for his discussion on the topic, referring back to chapter 7 if you find the macro definitions difficult to read.
Sure, macros are functions that take functions as input, and produce new functions as output. But they take the function's symbols as input and produce a new set of symbols. So a macro can extend the syntax of the language without having to modify the core language system. Anyway, what's unique about Lisp macros vs say, Rust macros, or C style preprocessors, is "homoiconicity". The data structure that a Lisp macro takes as an input, the lisp code, is the same data structure that the language uses normally (S-expressions, lists...), so writing a macro requires few new language skills compared to writing normal lisp (again, compare writing Rust macros, a dark art in comparison).
If it helps, https://taeric.github.io/CodeAsData.html was my attempt at exploring "code as data" and what makes lisp different here. My specific focus was more to point out that "eval" in lisp doesn't just take in a string. But I think the same general points remain.
1. Come up with an algorithm to define an algorithm.
2. Code expansion. Instead of typing out 1000 classes that are best represented as a template of a single class, you can define a macro then use it.
3. C++ at least uses them to provide generics.
4. They let you peel back the layers of abstraction to use the language itself as an API. Useful if you want to write static analysis to do analysis on code quality, security, linting, etc.
5. Anything you can imagine, it's metaprogramming.
There's a bit of a mental model flip to make maybe.
> they're code that's generated at compile time
They're code that generates code at compile time. Macros can actively walk the AST of the parameters they process and rewrite them completely into new shapes. That transformed AST is what then actually gets compiled.
one way to see it is that it's a function that runs at compile time. for instance instead of dumping magic numbers/tables in a codebase you could put the code and substitute to their value at compile time.
but also it can change your code, so you get to do all the java annotation magic stuff.
Macros are conceptually similar to FEXPRs in that they act like functions that don't evaluate their operands and return code. The (only?) difference is that macro are all expanded before execution/runtime.
Depends very much on your harness and effective use of tools. For vibe-coding, stick to Python.
It can get good with the right setup. I made Codex work with GToolkit (similarly underrepresented, but technically impressive and a seemingly good fit for LLMs), but it took a lot of tweaking of the project structure, extensive instructions in AGENTS.md, and some custom skills (some borrowed from Gt4Llm, the GT's built-in model harness). Out of the box, it burned tokens and took way too long to implement even basic things.
YMMV - as with everything LLM-related - but I think without a similar setup, an agent instructed to write CL/Clj/Scheme/Racket will have the same issues. It might be better in an established, large project - but starting from an empty Git repo, I suspect you'll have to fight your way to productivity.
I've found LLMs to be bad at balancing parenthesis. I've also found them to be less likely to hallucinate library types in dynamic languages, they tend to hallucinate arguments to library functions/methods instead.
Yeah but apart from SBCL, Viaweb, Hacker News, Emacs, Clojure, Scheme, Racket, garbage collection, macros, homoiconicity, the REPL, S-expressions, symbolic computation, what has Lisp ever done for us?
> [...] among its extensibility, its interactive environment, the REPL, and a lot of other features we haven’t touched yet. It is the combination of all of them that makes Lisp programming what it is.
Agreed. However, although the alternatives are few, they do exist. Today, I'd like to convince you (whether you're OP or a commenter) to give one of them a try. I'm talking about GToolkit[1]: Smalltalk/Pharo-based reimagining of Smalltalk as a productive environment for modern system design, analysis, and implementation. It's based on Pharo and its VM, but with GT-specific extensions and replacements, developed on GitHub in both Smalltalk and Rust.
I used both Common Lisp and Smalltalk over the years for some of my side projects. Technically, the environments are comparable: image-based, live, interactive development is central to both. Lisp is easier to fit into modern workflows: it's still just files on disk. If you ignore the REPL and treat the image as a bundler, you can have a Java- or Python-like development workflow. You can gradually adopt the more interactive ways of working with the codebase. Smalltalk was historically more of an "all or nothing" approach, but nowadays it supports Git-based workflows as first-class, with GToolkit providing additional tools directly. GT is tied to a single IDE, but that IDE is genuinely powerful, pragmatic, and easy to customize: the entire IDE is Smalltalk code[2] that lives in the image beside your code, so you can live-edit any part of it at any time.
The languages differ, most obviously in the object model (single inheritance and message sends in Smalltalk, and multiple inheritance with multimethods in CLOS), but I don't think one is strictly better than the other for the vast majority of code. You just need to structure your code differently. Both systems are very dynamic, so neither is like Java or C++. After quite a few projects in both, I am convinced that this difference doesn't matter.
Both languages have very simple syntax. Smalltalk doesn't have macros; instead, it has an extensible/replaceable compiler for method bodies. This capability is used, for example, for compiling grammars into parsers/lexers in SmaCC. It's not as convenient for control-flow abstraction as CL's macros, but Smalltalk tends to make them out of blocks (lexical closures with non-local exits) plus actions on thisContext. In practice, I never found either language too limiting in what I could express, and both have small, regular basic syntax.
TL;DR: Common Lisp and GToolkit Smalltalk offer comparable technical merits, with the most pronounced difference being GT's built-in, high-quality IDE, which open source Lisp lacks.
The IDE (and the capability to build cross-platform, but natively rendered GUI apps) is the major selling point of GToolkit, but the bigger reason to consider it is social. Lisp had a bit of a renaissance from 2005 to 2015, but it has since died down. A lot of great things for Lisp appeared in that time: ASDF and Quicklisp being prime examples. Unfortunately, the momentum was lost, and since then, the rate of development has plummeted. It's still a solid proposition because of its stability, which means a lot of old code still works perfectly well, but it's a double-edged sword: it would take a hundred miracles in a row for CL to get a plausible M:N concurrency story, for example. This isn't true for Smalltalk, and doubly so.
There's a small but active community around Pharo, mostly academics and hobbyists (IME; and it's from a decade ago, so YMMV). Pharo itself is already a "Smalltalk-like" language, and it regularly gets features that are not in original Smalltalk (stateful traits and slots (reification of class and instance variables) are good examples). It continues to evolve, and each version brings notable improvements to the language, the VM, and the class library. In CL, since there are many implementations maintained by various groups and the standard is set in stone, adopting extensions to the standard is incredibly hard and time-consuming. You can, of course, commit to a single implementation, but there are only 2 or 3 implementations that are actively developed and trying to do something "new"; they are all experimental and incomplete. And while SBCL has quite a few nice features, its codebase was a bit hard for me to grok (read: days on end of banging my head against a wall, then giving up; I'm much more comfortable messing with Smalltalk bytecode compiler).
Secondly, GToolkit is driven by a group of coders[3] who have established a profitable consultancy. It's being developed around a single vision that the authors swear helps them achieve their projects' goals. Even if you're not sold on moldable development[4] as a methodology, GToolkit gives you all the tools you might need in a modern development and then adds tons of domain-specific tools and utilities that might be useful after just a bit of adjustment. The whole environment is built to make such adjustments as painless as possible, too.
The only problem I experience with GToolkit is the lack of structured, book-like documentation. The GToolkit book is closer to the PHP wiki or a bundle of tutorials than something like the Rust book. It's not that the docs aren't there - they often are, but finding them effectively is a bit challenging. I found that simply cloning all the repos that GT consists of (tens, if not hundreds, but there's automation for that) and pointing an agent to the directory with them is often enough to quickly find what I'm looking for (if the built-in Spotter fails and I'm too lazy to construct an in-image search).
TL;DR: GToolkit/Pharo move much faster than the CL world, and while the number of maintainers might be similar (honestly hard to estimate, though), the effort on the GT side is more centralized and, in my opinion, heads in a better direction. If you want an extensible, live, dynamic environment that shares many of Common Lisp's strengths, but is more polished, more actively maintained, and feels more modern, take a look at GToolkit.
(Disclaimer: no affiliation, just a programmer who likes to explore unpopular languages).
There are many articles extolling the virtues of Lisp. I would like to see some articles that have a level-headed criticisms or critique of Lisp, it's ideas and it's place in the ecosystem of languages.
Articles like this, and the PG articles it references, amount to "if you know, you know". I understand the appeal and I understand the explicit and implicit arguments this article is making.
Computer programming has matured quite a bit in the past 60 years. I would like to see more articles that are more considered in their examination.
Programming is in tension between the Light Side and the Dark Side.
The Light Side is about preventing the programmer from making mistakes: Get rid of go-tos! Add static types! Do not allow a bug to be expressible. The Dark Side is about giving power to the programmer: Macros? Obviously. Operator overloading? Self-modifying code? Multi-line reg-exps? Go to town!
The Light Side knows programmers are flawed and imposes constraints. The Dark Side trusts programmers with power. Neither side is correct all of the time, and a good programmer learns both.
Lisp is interesting in that it is clearly Dark Side programming (the programmer can do anything) but it's still admired by Light Side programmers. Maybe there's something about the simplicity of the language that makes it seem platonic--almost incorruptible. Or maybe Lisp is so pure that it embodies both Light Side and Dark Side, like a god that spawned the programming universe.
Kind of agree, but in my view preventing the programmer to make mistakes is futile. I have seen awful stuff in languages made to prevent errors.
It's much better to give all the power to the programmer, to allow him to fix his mistakes rather than fantasising about preventing them.
> Or maybe Lisp is so pure that it embodies both Light Side and Dark Side, like a god that spawned the programming universe.
This isn't so far off, considering that some people consider the "Lisp in Lisp" bit from the 1.5 manual to be the "Maxwell's Equations in Software":
https://michaelnielsen.org/ddi/lisp-as-the-maxwells-equation...
Hah, didn't think of it this way... To me it was if it allows live updates (+support for them) or not :)
The website seems to have a bug with syntax highlighting. Pieces of code included in the post text are black, you can still see the actual text if you select it with your mouse. Same bug on Chrome desktop and on Safari on iPad
I get this on both FF and Chrome.
My first thought was that the article was redacted lol
metoo
There are some truly powerful and unique things about Lisps, but I wish articles like this would stop including REPLs and hot-reloading. The former have been table stakes for interpreted languages (and some compiled ones!) for years, and the latter is neither unique nor particularly widely used (hot reloads have to tangle with state and patching, so resetting the world for ease of reasoning is considered a best practice for a reason).
> The former have been table stakes for interpreted languages
I used to think so too back in the day when I was getting into Clojure. It was much later when I realized that when Lisp people talk about the REPL they’re usually talking not so much about the interactive CLI where you can evaluate commands easily but more so the ability to connect your program to a live session where you can quickly evaluate forms within your text editor and in the context of your running application, which enables much more interactive development than in other interpreted languages.
You've been able to do this in Ruby since I can remember. Not a lot of editors take advantage of it though...
Largely agreed.
I do think it is worth highlighting how many advanced parts of hot-reloading have already been covered in Common Lisp. Same with highlighting how the REPL is largely not used to directly type into, but is instead a very powerful interface for tools to interact with a running image.
But, again agreed that simply these existing are not that notable today.
But with Clojure and immutable by default, hot reload is a real thing, it sounds like not just on JVM but even among lisps.
I must admit - I still don't understand macros. I get that they're code that's generated at compile time. But I don't understand how that's different than a function which evaluates other functions. I guess the latter would actually be evaluated at runtime? I think I get it conceptually but I'm not sure I have the muscle memory to reach for them. Anybody here have an "ah hah!" Moment with macros?
If you haven't read it, I'd suggest taking a look at Paul Graham's book On Lisp [0]. He says better, and with more examples than I'd provide in a comment block, what I'd write on the subject. Jump to chapter 8 for his discussion on the topic, referring back to chapter 7 if you find the macro definitions difficult to read.
[0] https://www.paulgraham.com/onlisptext.html
Sure, macros are functions that take functions as input, and produce new functions as output. But they take the function's symbols as input and produce a new set of symbols. So a macro can extend the syntax of the language without having to modify the core language system. Anyway, what's unique about Lisp macros vs say, Rust macros, or C style preprocessors, is "homoiconicity". The data structure that a Lisp macro takes as an input, the lisp code, is the same data structure that the language uses normally (S-expressions, lists...), so writing a macro requires few new language skills compared to writing normal lisp (again, compare writing Rust macros, a dark art in comparison).
If it helps, https://taeric.github.io/CodeAsData.html was my attempt at exploring "code as data" and what makes lisp different here. My specific focus was more to point out that "eval" in lisp doesn't just take in a string. But I think the same general points remain.
You could use them to:
1. Come up with an algorithm to define an algorithm.
2. Code expansion. Instead of typing out 1000 classes that are best represented as a template of a single class, you can define a macro then use it.
3. C++ at least uses them to provide generics.
4. They let you peel back the layers of abstraction to use the language itself as an API. Useful if you want to write static analysis to do analysis on code quality, security, linting, etc.
5. Anything you can imagine, it's metaprogramming.
There's a bit of a mental model flip to make maybe.
> they're code that's generated at compile time
They're code that generates code at compile time. Macros can actively walk the AST of the parameters they process and rewrite them completely into new shapes. That transformed AST is what then actually gets compiled.
one way to see it is that it's a function that runs at compile time. for instance instead of dumping magic numbers/tables in a codebase you could put the code and substitute to their value at compile time.
but also it can change your code, so you get to do all the java annotation magic stuff.
Macros are conceptually similar to FEXPRs in that they act like functions that don't evaluate their operands and return code. The (only?) difference is that macro are all expanded before execution/runtime.
I thought this was going to be https://web.archive.org/web/20120106121645/http://wiki.alu.o.... Brings back memories of following comp.lang.lisp, not least as a sort of soap opera. Lots of characters and drama, as well as programming wizardry.
I've been wondering - Is lisp (common lisp, clojure, scheme) easier for iterative work with LLMs?
Depends very much on your harness and effective use of tools. For vibe-coding, stick to Python.
It can get good with the right setup. I made Codex work with GToolkit (similarly underrepresented, but technically impressive and a seemingly good fit for LLMs), but it took a lot of tweaking of the project structure, extensive instructions in AGENTS.md, and some custom skills (some borrowed from Gt4Llm, the GT's built-in model harness). Out of the box, it burned tokens and took way too long to implement even basic things.
YMMV - as with everything LLM-related - but I think without a similar setup, an agent instructed to write CL/Clj/Scheme/Racket will have the same issues. It might be better in an established, large project - but starting from an empty Git repo, I suspect you'll have to fight your way to productivity.
I've found LLMs to be bad at balancing parenthesis. I've also found them to be less likely to hallucinate library types in dynamic languages, they tend to hallucinate arguments to library functions/methods instead.
All roads lead to Lisp
What has Lisp ever done for us?
Flight search (Google's ITA Software), underground planning (SISCOG), CAD software (PTC Creo et all), the Gollum face (Mirai), automation of document extraction, renewable energies resource planning (3E), project management (planisware),
games (Kandria),
Maxima,
a faster pgloader (https://tapoueh.org/blog/2014/05/why-is-pgloader-so-much-fas...),
hackernews (was rewritten to SBCL: https://lisp-journey.gitlab.io/blog/hacker-news-now-runs-on-...), first reddit…
https://lisp-screenshots.org/
https://www.lispworks.com/success-stories/
also Emacs (for another dialect).
Yeah but apart from SBCL, Viaweb, Hacker News, Emacs, Clojure, Scheme, Racket, garbage collection, macros, homoiconicity, the REPL, S-expressions, symbolic computation, what has Lisp ever done for us?
The aqueducts!
> So why Lisp (or when)
> [...] among its extensibility, its interactive environment, the REPL, and a lot of other features we haven’t touched yet. It is the combination of all of them that makes Lisp programming what it is.
Agreed. However, although the alternatives are few, they do exist. Today, I'd like to convince you (whether you're OP or a commenter) to give one of them a try. I'm talking about GToolkit[1]: Smalltalk/Pharo-based reimagining of Smalltalk as a productive environment for modern system design, analysis, and implementation. It's based on Pharo and its VM, but with GT-specific extensions and replacements, developed on GitHub in both Smalltalk and Rust.
I used both Common Lisp and Smalltalk over the years for some of my side projects. Technically, the environments are comparable: image-based, live, interactive development is central to both. Lisp is easier to fit into modern workflows: it's still just files on disk. If you ignore the REPL and treat the image as a bundler, you can have a Java- or Python-like development workflow. You can gradually adopt the more interactive ways of working with the codebase. Smalltalk was historically more of an "all or nothing" approach, but nowadays it supports Git-based workflows as first-class, with GToolkit providing additional tools directly. GT is tied to a single IDE, but that IDE is genuinely powerful, pragmatic, and easy to customize: the entire IDE is Smalltalk code[2] that lives in the image beside your code, so you can live-edit any part of it at any time.
The languages differ, most obviously in the object model (single inheritance and message sends in Smalltalk, and multiple inheritance with multimethods in CLOS), but I don't think one is strictly better than the other for the vast majority of code. You just need to structure your code differently. Both systems are very dynamic, so neither is like Java or C++. After quite a few projects in both, I am convinced that this difference doesn't matter.
Both languages have very simple syntax. Smalltalk doesn't have macros; instead, it has an extensible/replaceable compiler for method bodies. This capability is used, for example, for compiling grammars into parsers/lexers in SmaCC. It's not as convenient for control-flow abstraction as CL's macros, but Smalltalk tends to make them out of blocks (lexical closures with non-local exits) plus actions on thisContext. In practice, I never found either language too limiting in what I could express, and both have small, regular basic syntax.
TL;DR: Common Lisp and GToolkit Smalltalk offer comparable technical merits, with the most pronounced difference being GT's built-in, high-quality IDE, which open source Lisp lacks.
The IDE (and the capability to build cross-platform, but natively rendered GUI apps) is the major selling point of GToolkit, but the bigger reason to consider it is social. Lisp had a bit of a renaissance from 2005 to 2015, but it has since died down. A lot of great things for Lisp appeared in that time: ASDF and Quicklisp being prime examples. Unfortunately, the momentum was lost, and since then, the rate of development has plummeted. It's still a solid proposition because of its stability, which means a lot of old code still works perfectly well, but it's a double-edged sword: it would take a hundred miracles in a row for CL to get a plausible M:N concurrency story, for example. This isn't true for Smalltalk, and doubly so.
There's a small but active community around Pharo, mostly academics and hobbyists (IME; and it's from a decade ago, so YMMV). Pharo itself is already a "Smalltalk-like" language, and it regularly gets features that are not in original Smalltalk (stateful traits and slots (reification of class and instance variables) are good examples). It continues to evolve, and each version brings notable improvements to the language, the VM, and the class library. In CL, since there are many implementations maintained by various groups and the standard is set in stone, adopting extensions to the standard is incredibly hard and time-consuming. You can, of course, commit to a single implementation, but there are only 2 or 3 implementations that are actively developed and trying to do something "new"; they are all experimental and incomplete. And while SBCL has quite a few nice features, its codebase was a bit hard for me to grok (read: days on end of banging my head against a wall, then giving up; I'm much more comfortable messing with Smalltalk bytecode compiler).
Secondly, GToolkit is driven by a group of coders[3] who have established a profitable consultancy. It's being developed around a single vision that the authors swear helps them achieve their projects' goals. Even if you're not sold on moldable development[4] as a methodology, GToolkit gives you all the tools you might need in a modern development and then adds tons of domain-specific tools and utilities that might be useful after just a bit of adjustment. The whole environment is built to make such adjustments as painless as possible, too.
The only problem I experience with GToolkit is the lack of structured, book-like documentation. The GToolkit book is closer to the PHP wiki or a bundle of tutorials than something like the Rust book. It's not that the docs aren't there - they often are, but finding them effectively is a bit challenging. I found that simply cloning all the repos that GT consists of (tens, if not hundreds, but there's automation for that) and pointing an agent to the directory with them is often enough to quickly find what I'm looking for (if the built-in Spotter fails and I'm too lazy to construct an in-image search).
TL;DR: GToolkit/Pharo move much faster than the CL world, and while the number of maintainers might be similar (honestly hard to estimate, though), the effort on the GT side is more centralized and, in my opinion, heads in a better direction. If you want an extensible, live, dynamic environment that shares many of Common Lisp's strengths, but is more polished, more actively maintained, and feels more modern, take a look at GToolkit.
(Disclaimer: no affiliation, just a programmer who likes to explore unpopular languages).
[1] https://gtoolkit.com/
[2] Some parts are implemented as Rust dynamic libraries and called from Smalltalk via FFI.
[3] https://feenk.com/
[4] https://moldabledevelopment.com/